With the right RPG tools, you can make the most challenging data manipulation project fun.

Dealing with interface files from external systems is always an involved exercise in data validation. It's also a prime opportunity for using the string manipulation built-in functions (BIFs) available in RPG ILE.

Suppose you receive from an external system a file that is not available in an XML format, so you are required to process an ASCII text file. You have to be able to handle it and process it, especially if it contains data that generates revenue. You don't want to reject or misinterpret a single record, so you need to process that data and convert it into a usable format.

Here are some string manipulation BIFs that we will be using in our data-processing example program:

• %TRIM(string)--This function returns a string that is the result of removing the leading and trailing blanks from the string passed in as the TRIM parameter.

• %TRIML(string)--This function returns a string that is the result of removing only the leading blanks from the string passed in as the TRIML parameter.

• %TRIMR(string)--This function returns a string that is the result of removing only the trailing blanks from the string passed in as the TRIMR parameter.

• %SUBST(string:start{:length})--%SUBST returns a portion of the string that is passed in as the parameter. It will start at the position indicated in the second parameter. If the third parameter is not specified, it will return the portion from the starting position to the end of the string. If the third parameter is specified, it will return the part of the string from the starting position to include the number of characters specified in the third parameter. The first character position of the string parameter has a starting index value of 1. You could also use the %SUBST function on the left side of the EVAL statement to set a specific set of bytes within the character string.

• %XLATE(from-characters: to-characters: string{: start position})--The %XLATE BIF will translate all instances of the from-characters specified in the first parameter into the to-characters specified in the second parameter. The value returned will have the character translations performed on the string that is passed in the third parameter. You can optionally specify the starting position of the translations.

• %CHECK(comparator: base {:start})--%CHECK is used to check the characters of the base string to make sure that all of the base characters are contained within the comparator. If all of the characters in the base are contained within the comparator, the result will be zero. If a character is found in the base that is not contained in the comparator, the position of the first character in the base that is not contained in the comparator will be returned.

• %CHECKR(comparator: base {:start})--%CHECKR isthe same as %CHECK except that it goes from right to left instead of left to right. This is good for finding unspecified characters from the back of the string.

Here is a simple program that emulates a worst-case file record read without the file input/output complexities to distract from the main objective of string manipulation. I've created a simple string to represent a read operation that acts as a single line of data from an external file.

D inputBytes      S             42A

D subBytes        S             10A

D numberBytes     S             10A

D junkInBytes     S             10A

D junkOutBytes    S             10A

D displayBytes    S             52A

D posi            S             10I 0

D*   XLATE STRUCTURE FOR LOWER TO UPPER CASE CONVERSION + X'3A' FOR E'*

D lc              S             27A 

D uc              S             27A  

 /free

  // Lowercase and Uppercase Characters

  lc = 'abcdefghijklmnopqrstuvwxyz';

  uc = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';

  // ALSO include conversion of Acute Accent "E'" to "E" in Cafe' ***

  lc = %TRIM(LC) + X'3A';

  uc = %TRIM(UC) + 'E';

  junkInBytes = ':"''-,';

  junkOutBytes = '     ';

  numberBytes = '1234567890';

  inputBytes = '   319 '

             + 'Caf' + X'3A'

             + ', "Double" ''Quotes'':'

             + ' 1208-0807    ';

  displayBytes = '[' + inputBytes + ']';

  DSPLY displayBytes;

In the string, you can see spaces at the beginning and end of the string, an acute accent e, double quotes, single quotes, a colon, and a hyphen between the last numbers just for good measure.

When %trim is executed, you will see that both the leading and trailing blanks were removed.

  // trim

  displayBytes = '%trim['

               + %trim(inputBytes)

               + ']';

  DSPLY displayBytes;

When %trimR is executed, only the trailing blanks are removed.

  // trimR

  displayBytes = '%trimR['

               + %trimR(inputBytes)

               + ']';

  DSPLY displayBytes;

When %trimL is executed, only the leading blanks are removed.

  // trimL

  displayBytes = '%trimL['

               + %trimL(inputBytes)

               + ']';

  DSPLY displayBytes;

Using a combination of %check and %trimL, we are able to retrieve the first numeric value, which could have possibly been passed as a number in a right-justified field.

  // Using Check to get the Front Number.

  posi = %check(numberBytes:%trimL(inputBytes));

  subBytes = %subst(%trimL(inputBytes):1:posi);

  displayBytes = 'Front # ['

               + %trim(subBytes)

               + ']';

  DSPLY displayBytes;

Using a combination of %checkR and %trimR, we can retrieve the last numeric value, which could have possibly been passed as a number in a left-justified field.

  // Using CheckR to get the Back Number.

  posi = %checkR(numberBytes:%trimR(inputBytes));

  subBytes = %subst(%trimR(inputBytes):posi+1);

  displayBytes = 'Back # ['

               + %trim(subBytes)

               + ']';

  DSPLY displayBytes;

Retrieving the numbers at both the front and back of the string, the %subst BIF was used to pull the specified characters from the main string.

The lc->UC line shows the %xlate BIF providing one means of converting all of the lowercase letters to uppercase, with an additional conversion for the acute accent e.

  // XLATE to UPPER Case

  displayBytes = 'lc->UC['

               + %xlate(lc:uc:inputBytes)

               + ']';

  DSPLY displayBytes;

The junk line shows the %xlate BIF in action again, removing any characters that you may find undesirable in your database when bringing the information in. Of course, junk is in the eye of the developer, so your list of undesirable characters may differ.

  // Strip the Junk

  displayBytes = 'junk['

               + %xlate(junkInBytes:junkOutBytes:inputBytes)

               + ']';

  DSPLY displayBytes;

And the clean line has the kitchen sink thrown at it, where all of the modifications were applied to the data to provide the intended clean results.

  // All Cleaned Up

  displayBytes = 'clean['

               + %trim(%xlate(lc:uc:

                 %xlate(junkInBytes:junkOutBytes:inputBytes)))

               + ']';

  DSPLY displayBytes;

  *inlr = *ON;

 /end-free

The Output

After running the program, you will see this output :

 > call mcpress

     DSPLY  [   319 Caf , "Double" 'Quotes': 1208-0807 ]

     DSPLY  %trim[319 Caf , "Double" 'Quotes': 1208-0807]

     DSPLY  %trimR[   319 Caf , "Double" 'Quotes': 1208-0807]

     DSPLY  %trimL[319 Caf , "Double" 'Quotes': 1208-0807 ]

     DSPLY  Front # [319]

     DSPLY  Back # [0807]

     DSPLY  lc->UC[   319 CAFE, "DOUBLE" 'QUOTES': 1208-0807 ]

     DSPLY  junk[   319 Caf    Double   Quotes   1208 0807 ]

     DSPLY  clean[319 CAFE   DOUBLE   QUOTES   1208 0807]    

When the strings are displayed, they are surrounded by brackets to identify any leading and trailing blanks. In the final line of the output, you will see the results of applying all of the string manipulations, ending up with only the shiny, happy characters that will be entered into the database.

You may laugh, but I have actually had files that contained every one of these instances in a single file! No lie! But I am not one to say that it can't be done, so I did what I had to do.

Download the Code

You can download the code used in this article--as well as the fixed-format version--by clicking here.

The Right Tools for the Job

With the right RPG tools, you can take the most challenging data and make a fun project out of it.