Learn critical cleanup techniques that target end-of-job due to end job commands.

Author's note: This article is one of a series of articles to memorialize Simon Coulter, an outstanding IBM i expert who contributed much to the prosperity of the IBM i platform. I thank Gwen Hanna, Simon's partner, for providing the biography of Simon at the end of the first article of "Simon's Solutions."

As one the most robust operating systems and application systems in the world, IBM i provides numerous mechanisms to automatically reclaim resources occupied by resource containers, such as activation groups (ACTGRPs) or jobs that end unexpectedly. However, it's still possible for resources to be left as orphans by a job that ends unexpectedly—for example, a pointer-based mutex created by the job with the keep-valid option set to true, or a permanent MI object created without inserting its addressability into a context object (library). It's also bad that an unexpected end-of-job might damage the integrity of application data.

As you might have seen, one of the most common reasons for an unexpected end-of-job is end-job operations initiated by operators. One of Simon's posts in December, 2011 covered most of the cleanup techniques that can be used to monitor for end-job operations. These techniques are based on different rationales and mechanisms, and studying them could help you understand the platform better.

In December 2011, Michael Smith started a discussion in the midrange-l mailing list on techniques that can be used to monitor for end-job operations. Simon listed a range of methods that can be used to achieve the goal in this post, Re: Condition Handlers and CL:

The only ways I know to monitor for ENDJOB/ENDSBS/ENDSYS are:

1/ Signal handler - see the signal handler API section of the Unix API reference manual. Specifically, the sigaction() function to monitor for the SIGTERM signal.

2/ Invocation exit programs - see the C runtime and MI library reference manuals. atexit() is used to specify a procedure that will be invoked during normal end. atiexit() is used specify a program that will be invoked during abnormal end.

3/ ILE APIs - see the Activation Group and Control Flow APIs in the ILE CEE API manual. Specifically, the CEERTX and CEE4RAGE APIs. The handler registered with CEERTX is invoked for ENDJOB *IMMED, and is invoked after the controlled delay time elapses for ENDJOB *CNTRLD. It also gets invoked for any exit except a return.

4/ Scope messages - see the message handler API manual. Specifically, the QMHSNDSM API (as mentioned in other threads). If you send the scope message to *EXT you can monitor for ENDJOB etc.

Since ending a job does not raise an exception in that job you cannot use the condition handler APIs.

Regards,

Simon Coulter.

Before Getting Started

At the MI level, an active IBM i job has an associated MI process that is identified by a Process Control Space MI object. The lifetime of an MI process consists of three phases: the initiation phase, the problem phase, and the termination phase. Each phase is initiated by invoking the corresponding phase program. It might surprise you a little that an MI process does the works it's expected to achieve in the problem phase. The system pointers to the initiation phase, problem phase, and termination phase program of an MI process can be materialized via the Materialize Process Attributes (MATPRATR) MI instruction with option hex 19, hex 1B, and hex 1A, respectively. A tiny RPG program, t180.rpgle, which materializes the name of the problem phase program of an MI process, is available in Appendix A for convenience.

An MI process could end for the following internal termination reasons:

• Return from first invocation in problem phase
• Return from first invocation in initiation phase and no problem phase program specified
• Terminate Thread MI instruction issued against the initial thread by a thread in the process
• Terminate Process MI instruction issued by a thread within the process
• An unhandled signal with a default signal-handling action of terminate the process or terminate the request was delivered to the process
• Exception was not handled by the initial thread in the process

And an MI process could end for the following external termination reasons:

• Terminate Process MI instruction issued explicitly to the process by a thread in another process
• Terminate Thread MI instruction issued explicitly to the initial thread of the process by a thread in another process

The Terminate Process and Terminate Thread MI instructions are blocked and cannot be issued from a User State program. At the OS level, a user terminates an active IBM i job via the end job commands: End Job (ENDJOB), End Subsystem (ENDSBS), End System (ENDSYS), and Power Down System (PWRDWNSYS). Each of these end job commands has a pair of similar parameters: How to end (OPTION) and Controlled end delay time (DELAY). The OPTION parameter specifies whether the job ends immediately or in a controlled manner that lets the application program perform end-of-job processing. In either case, the system performs certain job cleanup processing. When OPTION(*IMMED) is specified, the target job ends immediately and the system performs end-of-job cleanup. When OPTION(*CNTRLD) is specified, the job ends in a controlled manner by allowing the job to perform cleanup within the amount of time specified by the DELAY parameter (in seconds). If the cleanup is not completed before the end of the delay time, the job is ended immediately. (Only system cleanup is performed.) In either case, an end job command issued to an active job causes a SIGTERM asynchronous signal to be delivered to the target job if the job is enabled for signals and has a signal handler registered for SIGTERM. After the signal handler registered for SIGTERM receives control, the job has following amount of time to process cleanup works before finally being ended:

• Number of seconds specified by the DELAY parameter when OPTION(*CNTRLD) is specified
• Number of seconds specified by the QENDJOBLMT system value when OPTION(*IMMED) is specified

Details about monitoring for end job operations via the asynchronous signal support are discussed in section Monitor for End Job Operations by Catching the SIGTERM Asynchronous Signal.

It is important to keep in mind that once the signal handler for SIGTERM of a job receives control (or in other words, the SIGTERM asynchronous signal is caught successfully), the MI process associated with the job isn't regarded as being terminated externally; therefore, cleanup mechanisms targeting abnormal end-of-job (e.g., invocation exit procedures registered by CEERTX in order to monitor for abnormal end-of-invocations) will not be fired.

Besides monitoring for the SIGTERM signal, another way to know that a job is currently being ended by an end job operation is by retrieving the process status indicators via the MATPRATR MI instruction with option hex 20 in a registered invocation exit, activation exit, or scope program. This will be discussed in detail in Part II of this article.

For a job being ended by an end job command, to know the value of the OPTION parameter of an end job command issued to the job, you can check the 1-byte Controlled Cancel field in the Language/Utility Work Area (LUWA) at offset hex 10. A value of '0' indicates OPTION(*IMMED), and a value of '1' indicates OPTION(*CNTRLD), or in other words, a controlled cancellation of the current job is being performed. I discussed the LUWA in my article "How Much Do You Know About Job Switches?" The system built-in _LUWRKA returns the space pointer addressing the current job's LUWA. The Controlled Cancel field is referred to as End Status in CL and Work Management APIs documentation and can also be retrieved via the Retrieve Job Attributes (RTVJOBA) command with the ENDSTS parameter or the Retrieve Job Information (QUSRJOBI) API with format JOBI0600. For your convenience, examples of retrieving the Controlled Cancel field (End Status) via the three methods are listed below.

t179.rpgle

eoj01.clp

eoj02.rpgle

Be aware that the LUWA approach can not only retrieve but also change the Controlled Cancel field (End Status). For example, you can change the Controlled Cancel field via t179.rpgle and check it via eoj01.clp or eoj02.rpgle, like the following:

You can also change the Controlled Cancel field to '1' using t179.rpgle and then check the output of DSPJOB OPTION(*STSA) (or WRKJOB OPTION(*STSA)). The Controlled end requested field would look like the following:

Monitor for End Job Operations by Catching the SIGTERM Asynchronous Signal

Signals are a POSIX-defined Interprocess Communication (IPC) mechanism. POSIX.1 defines a "signal" as a mechanism by which a process may be notified of, or affected by, an event occurring in the system. Please refer to Using Signal APIs for detailed information about Signal concepts and Signal Management on IBM i. As mentioned above, when users issue an end job command to an IBM i job, a SIGTERM asynchronous signal is delivered to the target job. For this reason, monitoring for the SIGTERM signal with a signal handler registered for SIGTERM via the sigaction() API is a workable way to monitor for end job operations.

RPG Example of Monitoring an Asynchronous SIGTERM Signal

Signal APIs are ILE procedures exported by service program (*SRVPGM) QSYS/QP0SSRV1 and therefore can be utilized by all ILE High-Level Languages (HLLs). The following is an ILE RPG example, eoj03.rpgle, that demonstrates the steps of catching the SIGTERM asynchronous signal.

Submit a batch job that runs program EOJ03:

Then end the submitted job by issuing the ENDJOB *IMMED command. Messages sent to the QSYSOPR message queue might look like the following:

Pass Control Back from the Signal Handler to the Main Procedure/Program Directly

In the above RPG example, a global static indicator variable bye is used to communicate between the main procedure and the signal handler. The signal handler oops notifies the main procedure that a SIGTERM signal is caught by setting on bye. The main procedure checks the value of bye periodically. Once bye is set on, the main procedure completes necessary end-of-job cleanup and quits. An alternative method is to directly pass control back to the main procedure from inside the signal handler via a non-local goto:

1. Save the stack environment in the main procedure (or a procedure that is expected to retain control after a SIGTERM signal is caught) via the setjmp() or sigsetjmp() API.
2. When a SIGTERM signal is caught successfully inside the signal handler registered for SIGTERM, restore the saved stack environment via the longjmp() or siglongjmp() APIs so that the main procedure receives control and runs from the HLL statement where setjmp() is invoked.

All these four functions are declared in <setjmp.h> (QSYSINC/H.SETJMP). The longjmp() function is an ILE procedure exported by *SRVPGM QSYS/QC2UTIL1. The setjmp() function is implemented as a system built-in (__setjmp) instead of an ILE procedure, and the RPG prototype for __setjmp and structure jmp_buf_t can be found in mih-undoc.rpgleinc.

Note that, in addition to saving the current stack environment, sigsetjmp() can optionally save the current signal mask (the set of blocked signals of the calling thread). The stack environment and signal mask saved by sigsetjmp() can subsequently be restored by siglongjmp().

Benefits of the setjmp/longjmp (or sigsetjmp/siglongjmp) approach include these:

• End-of-job cleanup work can be started without any delay once a SIGTERM signal is caught.
• The signal handler can pass control to any invocation entry (that has been recorded via setjmp() in the saved stack environment) currently available on the invocation stack (aka call stack)—for example, the caller program of the program that registered the signal handler.

The following are parts of the source code of two RPG examples that show how to pass control back to a program's caller program from a signal handler via the setjmp/longjmp approach. Program EOJ05 (eoj05.rpgle) saves the stack environment in structure of type jmp_buf_t and then calls EOJ06 (eoj06.rpgle) and passes the jmp_buf_t structure. EOJ06 registers a signal handler for SIGTERM. When a SIGTERM signal is caught, the signal handler restores the stack environment via jmp_buf_t structure passed to EOJ06 and lets EOJ05 receive control again. Note that setjmp() (sigsetjmp()) returns 0 if returning directly; if the setjmp() (sigsetjmp()) returns as a result of a longjmp() (siglongjmp()) call, it returns the value argument of the longjmp() function, or 1 if the value argument of the longjmp() function is 0.

EOJ05

EOJ06

Submit a batch job that runs EOJ05 via a Submit Job (SBMJOB) command and then end it:

The output in the QSYSOPR message queue might look like the following:

Finally, if you have critical work that you don't want interrupted by an asynchronous SIGTERM signal, you can use the sigprocmask() (Examine and change blocked signals) API to block the SIGTERM signal temporarily and unblock it after you get the work done. The following example code is extracted from eoj04.rpgle.

Monitor for End-of-ACTGRP via ACTGRP Exit Programs/Procedures

The end of an IBM i job causes all activation groups (ACTGRPs) within the job to end. So monitoring the end of an ACTGRP is a good way to ensure resources scoped to ACTGRP are being released properly. Simon mentioned three methods to register ACTGRP exit programs/procedures in the post: the Register Activation Group Exit Procedure (CEE4RAGE/CEE4RAGE2) APIs and the C library routines atexit() and atiexit() exported by *SRVPGM QSYS/QC2UTIL1.

• atexit() registers procedures that are called when an ACTGRP ends normally.
• atiexit() registers programs that are called when an ACTGRP ends abnormally
• CEE4RAGE/CEE4RAGE2 registers procedures that are called when an ACTGRP ends (either normally or abnormally).

The only difference between CEE4RAGE and CEE4RAGE2 is that the first parameter of an ACTGRP exit procedure registered by CEE4RAGE is a 4-byte ACTGRP mark, while CEE4RAGE2 is an 8-byte ACTGRP mark.

The atiexit() function is documented in the ILE C/C++ for AS/400 MI Library and is declared in <mipgexec.h>, aka source member QSYSINC/H.MIPGEXEC. Prototypes of CEE4RAGE/CEE4RAGE2 and related structures are defined in <leenv.h> (QSYSINC/H.LEENV), and <letype.h> (QSYSINC/H.LETYPE). The prototype of the atexit() function can be found in <stdlib.h> (QSYSINC/H.STDLIB).

Activation group exit procedures, registered by CEE4RAGE, are called in the reverse order of their registration. If a procedure fails, subsequent procedures will not be called. The ILE C/C++ Run-Time Library Functions manual recommends that, for portability, you should use the atexit() function to register a maximum of 32 functions. The functions are processed in a last-in, first-out (LIFO) order. The atiexit() function can register only one ACTGRP exit program. If there are multiple calls to atiexit(), the last call to atiexit() takes effect.

It is important to note that none of these can be used in the default ACTGRPs!

In the following RPG example, eoj07.rpgle, all three methods to register ACTGRP exits mentioned above are utilized so that you can observe the behaviors of these cleanup mechanisms that target end-of-ACTGRP. Call EOJ07 with the one-character parameter 'a' or 'A' to end the ACTGRP abnormally before EOJ07 returns. (Program EOJ07 uses a named ACTGRP called EOJ07; see the control specification of eoj07.rpgle.) The Abnormal End (CEE4ABN) ILE CEE API is used in eoj07.rpgle to end the ACTGRP abnormally. To end ACTGRP EOJ07 normally after calling EOJ07, you can use the RCLACTGRP command: RCLACTGRP EOJ07 OPTION(*NORMAL). To end ACTGRP EOJ07 abnormally after program EOJ07 returns, issue the RCLACTGRP command with OPTION(*ABNORMAL): RCLACTGRP EOJ07 OPTION(*ABNORMAL).

The source of the ACTGRP exit program EOJ08, eoj08.clp, is the following. It simply sends the character string that is passed to it to the QSYSOPR message queue.

Call EOJ07 three times as shown:

You might get the follow messages in the QSYSOPR message queue:

Appendix A: RPG Example of Retrieving Problem Phase Program of the Current Job

ILE RPG program t180.rpgle retrieves a system pointer addressing the problem phase program of the current job via the Materialize Process Attributes (MATPRATR) MI instruction with option hex 1B and then materializes the context name and object name of it via the Materialize System Object (MATSOBJ) MI instruction.

To retrieve the problem phase program of a common batch job, you can submit a job to the QBATCH subsystem that calls program T180:

The message sent to the QSYSOPR message queue by T180 might look like this:

Or suppose you submit a job to the QBATCH subsystem with routing data 'QCMD38' that calls program T180:

The message sent to the QSYSOPR message queue by T180 might look like this: