Anyone who has worked in IS for any length of time becomes familiar with Murphy’s Law, “Anything that can go wrong will go wrong.” Unfortunately, Murphy’s Law usually rears its ugly head when you’re already up to your neck in other problems. With the AS/400, those problems can range from a user who can’t find a report to a system crashing right in the middle of a major batch process. If you haven’t planned for the unexpected, you could be left high and dry after the tide of disaster washes your business out to sea. This article examines some things you can do in your shop to improve the odds so you won’t waste time searching for your paddle the next time disaster strikes. It also provides an example utility you can use to recover your transaction-based processes. From a recoverability standpoint and depending on the type of business your AS/400 supports, you probably have some batch transaction-processing jobs that could be improved. The techniques in this article can help you recover gracefully from almost anything Murphy throws your way.

What’s a Transaction-based System, Anyway?

A transaction-based system processes units of work through a well-defined cycle in the most efficient manner possible. Systems that control bank deposits, shipments, and invoices are all transaction-based. For a system to be classified as transaction-based, its information process must move through a predictable cycle. Consider the following example: A customer transmits an order for merchandise to your company via electronic data interchange (EDI). After the electronic order is received, it goes through an approval, or edit, process. If the order passes all the edits, it then goes to the warehouse, where it is packed and shipped. The client receives a bill for the goods, and the order information is saved in a history database. The life cycle of this process is a well-defined series of transactions.

Regardless of the type of process your transactions go through, the types of programs involved are fairly standard. The typical batch job consists of multiple gathering, editing, and formatting programs that conclude with a print-and-update function that prints output and updates files. Transactions are grouped together in a batch to go through a particular process to increase efficiency of both the computers and the humans dealing with

the computers. Batch processes may involve printing a variety of documents, but many AS/400 shops don’t have the luxury of a printer for each form type. In addition, many of us still load and print the old tractor-fed forms, so it makes sense to batch transactions to reduce the number of times forms must be changed. In a perfect world, forms would never jam in the printer, and EDI transmissions would never be lost in cyberspace, but this isn’t a perfect world!

Some processes, such as check writing, have controls for how and when they can be run (or rerun). For many processes, however, you need to build in the ability to rerun batches through jobs without updating files or roll back the data to a point where you can rerun batches and re-update files. Given the backlog that many small shops face, issues relating to backup and recovery often go ignored during the development phase. You must eventually deal with these situations, but what is the best method for doing so? That depends on whether you are in the midst of designing a new system or retrofitting jobs so they can be recovered. Here are two methods you should consider for backup and recovery.

Commitment Control

Commitment control is a series of commands that, after some setup, let you define and process changes to database files in a single unit. Commitment control ensures that transactions are logically synchronized if a job ends abnormally. However, such a helpful tool has its price: Your files must be journaled. Journaling a file records all data changes to database files, so there is a significant disk space usage consideration in using it. When a job running under commitment control ends abnormally, the system uses the journals to roll back incomplete transactions to a state in which all databases are synchronized. The detailed application and program knowledge required to back out transactions must be built into the system, but once that knowledge is there, an operator’s recovery from an abnormal end is assured. The system automatically rolls back uncommitted transactions.

Now, what if the job ends normally but you accidentally delete the invoices from the output queue before you print them? If the COMMIT operation has taken place, you can still roll back the committed changes by using the Remove Journal Changes (RMVJRNCHG) command. This solution, however, is not automatic. The range of journaled changes you wish to remove must be known, specified, and set up ahead of time by the programmer or designer. As you can see, commitment control requires a significant investment of human and system resources up front. If being able to completely back out your transactions is particularly important in your application, you should consider using commitment control to improve recoverability.

An Alternate Approach to Recovery

Sometimes, the quick-and-dirty method is more appropriate, especially if you are making improvements to an existing system. One client of mine saved batches to an old System/36 diskette magazine before processing. As technology changed, I adapted this function to the newer methods available. Today, some of these same jobs still run on an AS/400, but programs now copy the batches to multiple-member files in a special library named BKUPDTA rather than to diskette. I use a program much like BACKUPCL, which is shown in Figure 1. This program is called from whatever batch processes on the system require backup. You pass to BACKUPCL the library and file name to be saved as well as the file in which you want them saved. Saving batches before doing any file updates does not allow you to automatically reverse any partially posted transactions due to an abnormal job end but ensures that you know exactly what is in each batch for later problem solving and analysis, and allows you to rerun jobs to reproduce output that has been lost or destroyed.

Figure 2 lists the objects you need to create and use this routine and a sample call of the program. Library BKUPDTA must exist with at least one multiple-member physical file

whose name begins with “BKU.” BKU files may be externally described with one field name that is as long as the longest record length in the batches you are saving, or you can simply use the Create Physical File (CRTPF) command and specify the record length. The file isn’t used directly in any programs but is a repository for members of (possibly) different record layouts, so whether it is externally described is not particularly important.

The third parameter passed provides the rest of the BKU file name and the prefix for the member name created. If the BKU file name constructed based on passed parameters doesn’t exist, the program uses the Create Duplicate Objects (CRTDUPOBJ) command to copy the batch in library BKUPDTA and alerts the operator that the file needs to be created. If this happens, CRTDUPOBJ still backs up the file; the file just does not exist in a member in the BKU file. Otherwise, the file copies the batch file to a member in the BKU file using the Copy File (CPYF) command and specifying FMTOPT(*NOCHK). This way, you can save multiple types of batches in the same file.

To create the member name, the program takes the first two characters of the &TYPE parameter and appends the system time to it. If this member already exists (which is unlikely unless someone forgets to do the special save described below for BKUPDTA), the program waits for one second, constructs a new member name based on the type and time, and tries again. The program is compiled with the USRPRF parameter equal to *OWNER. This mitigates any authority problems that might arise in the CRTDUPOBJ part of the program, assuming that the person compiling the program has authority over any file to be saved.

Special Backup Instructions

At this shop, we have set up multiple files in library BKUPDTA with names like BKUINVF for invoices and BKUAPF for accounts payable. Every night, the library is saved to a special tape with the date as part of the volume label information. At this time, all the members created for that day’s work are removed from the physical files after confirming that the save has been performed successfully. In addition, the library is backed up with all other data. This makes retrieval easy because at least a month’s worth of data fits onto one tape. All an operator needs to do to rerun a process is to locate the tape containing the appropriate member and restore library BKUPDTA for that day from tape (if necessary). Once the library has been restored, the data is copied to the batch file name using the FMTOPT( *NOCHK) option in the copy command.

The most common problem in this shop is losing or destroying printed output in printing. Because the original print/update function has already made appropriate file updates, we need a way to get the printed output again but not update the files. To handle this situation, many of our batch-processing routines accept a parameter that tells them not to do any file updates but simply to print the output. For instance, we can reprint a lost batch of invoices without updating an invoice date or adding a record to the accounts receivable file.

Saving the batches is also invaluable for debugging. The utility was set up to save batches for rerunning, but we also use it to save all the data we receive from and send to business partners. Being able to go back to the raw data allows us to investigate more thoroughly any issues that may arise between the two systems.

Don’t wait until Murphy calls again to protect your batch processes from unexpected failures. It will be sooner than you think!

Related Material

OS/400 Backup and Recovery V4R4 (SC41-5304-03, CD-ROM QB3ALE03)

/* PGM: BACKUPCL */
/* COMPILED WITH OWNER (QPGMR) AUTHORITY */

/* THIS PROGRAM IS CALLED FROM POSTING PROCEDURES TO ADD */

/* MEMBERS TO FILES CONSTRUCTED TO SAVE POSTING DATA. THESE */

/* FILES ARE BACKED UP NIGHTLY & THEN CLEARED. THE FILE NAME IS */

/* CONSTRUCTED FROM THE PREFIX ‘BKU’ AND THE TYPE CODE PASSED */

/* INTO THE PROGRAM. */

PGM PARM(&INAME &FLIB &TYPE)

DCL VAR(&FLIB) TYPE(*CHAR) LEN(10) /* library +

where file resides */

DCL VAR(&INAME) TYPE(*CHAR) LEN(10) /* file to +

be saved */

DCL VAR(&MEMBER) TYPE(*CHAR) LEN(10) /* member +

name to copy data to */

DCL VAR(&PREFIX) TYPE(*CHAR) LEN(2) /* prefix of +

member name */

DCL VAR(&TYPE) TYPE(*CHAR) LEN(7)

DCL VAR(&TIME) TYPE(*CHAR) LEN(8)

DCL VAR(&SAVFNAME) TYPE(*CHAR) LEN(10) /* which +

file to save it in */

/* CONSTRUCT FILE NAME TO SAVE TO FROM TYPE PASSED IN */

CHGVAR VAR(&SAVFNAME) VALUE(‘BKU’ *TCAT &TYPE)

/* GET PREFIX FOR MEMBER NAME FROM TYPE PASSED IN */

CHGVAR VAR(&PREFIX) VALUE(%SST(&TYPE 1 2))

/* CHECK TO BE SURE FILE EXISTS */

/* IF NO FILE TO ADD MEMBER TO, SEND MSG TO PGMR & SAVE FILE AS IS *

CHKOBJ OBJ(BKUPDTA/&SAVFNAME) OBJTYPE(*FILE)

MONMSG MSGID(CPF9801) EXEC(DO)
SNDMSG MSG(‘&SAVFNAME not found in bkupdta’) + TOUSR(*SYSOPR)

CRTDUPOBJ OBJ(&INAME) FROMLIB(&FLIB) OBJTYPE(*FILE) +

TOLIB(BKUPDTA) DATA(*YES)

GOTO GETOUT

ENDDO

AGAIN:

/* CONSTRUCT MEMBER NAME BASED ON PREFIX PASSED IN, AND TIME OF DAY*/

RTVSYSVAL SYSVAL(QTIME) RTNVAR(&TIME)

CHGVAR VAR(&MEMBER) VALUE(&PREFIX *CAT &TIME)

/* IF FILE EXISTS, CHECK TO SEE IF MEMBER EXISTS. IF NOT, CREATE

CHKOBJ OBJ(BKUPDTA/&SAVFNAME) OBJTYPE(*FILE) MBR(&INAM

MONMSG MSGID(CPF9815) EXEC(DO)

ADDPFM FILE(BKUPDTA/&SAVFNAME) MBR(&MEMBER)

CPYF FROMFILE(&FLIB/&INAME) +
TOFILE(BKUPDTA/&SAVFNAME) TOMBR(&MEMBER) + MBROPT(*ADD) FMTOPT(*NOCHK)

GOTO GETOUT

ENDDO

/* MEMBER EXISTED, TRY AGAIN

DLYJOB DLY(1) /* wait for time to change */

GOTO CMDLBL(AGAIN)

GETOUT: ENDPGM

Figure 1: BACKUPCL is a CL program that copies batch files before any updates or output takes place.

Objects needed to use BACKUPCL utility:
1. Library BKUPDTA
2. Nonexternally described physical file(s) that begin with the letters "BKU"

Sample call to BACKUPCL utility: CALL PGM(BACKUPCL) PARM('FILE2SAVE ' 'DATALIB ' 'INVF ')
The above would create a backup member copying FILE2SAVE from library DATALIB in the file BKUINVF.

Figure 2: BACKUPCL needs these objects to run. Also, here’s a sample call of the program.