Learn the ins and outs of level-checking.

In IBM i, high-level language (HLL) programs are dependent on receiving, at run time, an externally described file (a database file, or a device file, such as a display file, or a printer file) whose format agrees with what was copied into the program at compilation time. For this reason, the system provides a level-checking function that ensures that the format is the same. Level-checking occurs on a record-format basis when the file is opened unless you specify LVLCHK(*NO) when you issue a file override command or create a file. If the level-check values do not match, the program is stopped by a level-check error, aka the CPF4131 error message.

CPF4131 might be one of the most frequently encountered error messages for a novice RPG programmer. Level-check errors occurring in a production environment may suggest poor change management. However, as an experienced IBM i programmer, you should know the answers to the following questions related to the level-checking function: 

• What is actually being checked when level-checking is performed?
• The system supplies the level-checking function for database files and device files. How does a program indicate to the system whether level-checking is to be performed when it opens a file?
• How is compatibility achieved across multiple versions of a database file by using LVLCHK(*NO)?

What Is Actually Being Checked When Level-Checking Is Performed?

The system assigns a unique level identifier for each record format when the file it is associated with is created. The system uses the information in the record format description to determine the level identifier. This information includes the total length of the record format, the record format name, the number and order of fields defined, the data type, the size of the fields, the field names, the number of decimal positions in the field, and whether the field allows the null value. Changes to this information in a record format cause the level identifier to change. In other words, the system determines the record format-level identifier (RCDFMT-ID) according to the record format itself; information not related to the record format is not taken into consideration—for example, key-field specifications, select/omit attributes, or join specifications. And what is actually being checked at runtime is the record format-level identifier, which identifies a record format uniquely.

At the MI level, record formats are represented by a specific type of permanent space object in the system domain: the record format (*FMT) object (with MI object type/subtype code hex 1951). *FMT objects are one of all the MI object types that comprise a file object at the OS level. System pointers to the *FMT objects referenced by a file object can be found in the associated space of the hex 1901 *FILE object of a file object. At V5R4, a system pointer (SYSPTR) addressing the *FMT object associated with a physical file can be found at offset hex 0380 into the hex 1901 *FILE object's associated space.

If you're interested in what's being stored in a *FMT object, you can select a file object, dump the content of it via a dump command (DMPOBJ or DMPSYSOBJ) or the System Service Tools (SST), find the virtual address of the *FMT object(s) referenced by the file in the dump result, and then dump the content of the *FMT object(s) via SST by the virtual address of the *FMT object. For example, following these steps, you can get the content of the *FMT object storing the record format description of a physical file (PF) called A1:

First, dump the content of PF A1 via the DMPOBJ command: DMPOBJ A1 *FILE. Here's the DDS source code of PF A1:

Now, find the system pointer to the *FMT object referenced by A1 at offset hex 0380 in A1's associated space. In this example, the value of the system pointer is hex 0000000000000000 2EA46D8979001900. Copy the 5-byte segment identifier (SID), which is the first 5 bytes of the lower-order 8 bytes of the system pointer, to perform the next step. In a system pointer, this 5-byte SID is the base segment ID of the object addressed by the system pointer. The 8-byte Single-Level Store (SLS) virtual address of a *FMT object is the base segment ID followed by 3-byte offset value hex 000000.

Start a SST session via the STRSST command and choose the following menu items:

At the "Find By Object Address" screen, enter the SLS virtual address of the *FMT object as shown below and press Enter to locate the *FMT object.

At the "Select Format" screen, chose menu item 2, Base structure to dump the content of the *FMT object. The dump result might look like this:

Note: The above example dump result is retrieved at VRM540, as an internal object type. The format of *FMT objects would possibly change in newer releases.

Now, you can investigate the format of the record format description information stored in the *FMT object. For example, the header portion seems to have a fixed length of hex 5F bytes, and the meanings of some of the fields in the header portion can be figured out easily:

Following the header portion are variable-length record-field entries, each starting with a BIN(2) length field that indicates the length of the current record-field entry.

How a Program Tells the System Whether to Perform Level-Checking When Opening a File

The system supplies the level-checking function for database files and device files. HLL compilers take advantage of the function to make sure record formats of externally described files found at runtime by an HLL program agree with what the compiler saw when the program was compiled. This way, an HLL program can be stopped as soon as an unexpected record format is detected; therefore, more serious errors due to the program receiving the wrong data content can be avoided. Also, there are conditions in which HLL compilers do not force level-checking upon file-opening; program-described files in COBOL and RPG are an example, and C library routine _Ropen() never takes level-checking into consideration.

Whether level-checking is performed when a file is being opened is determined by the following rules:

1. 1.When a file created with parameter LVLCHK(*NO) is opened by an HLL program, the system does not perform level-checking.
2. 2.When a file whose level-check attribute is overridden by an override command (such as OVRDBF or OVRDSPF) with parameter LVLCHK(*NO) is opened by an HLL program, the system does not perform level-checking.
3. 3.When opening a file, if a program indicates not to perform level-checking via the User File Control Block (UFCB) through which it accesses the file, level-checking is not performed.
4. 4.In all other conditions, level-checking is performed by comparing each RCDFMT-ID of the file to be opened with the corresponding RCDFMT-ID supplied by the requesting program in the UFCB.

The User File Control Block (UFCB)

What is UFCB? The following definition of UFCB is quoted from Chapter 12, Input/Output in MI Using the SEPT, of a great e-book, AS/400 Machine-Level Programming, written by Leif Svalgaard:

Strictly speaking a file is not an MI concept. At the MI-level, data is stored in spaces, not files. Files are opened, read/written, then closed. At the MI-level there is no concept of opening/closing of files. Files are defined at the OS/400 level (CPF) above the MI. A program (even an MI-program) accesses a file through a User File Control Block (the UFCB). The UFCB defines the filename, library name, possibly a member name, buffer areas and all necessary other control information needed to manage the file. It also provides pointers to the various feedback areas and access to various control structures, such as the Open Data Path (the ODP). The main purpose of the UFCB is to provide device independence for I/O. You use a UFCB for database files, display files, spool files, save files, tape files, etc. The fact that these files often have very different internal structure (the actual objects making up the files) is transparent to the programmer.

The UFCB consists of a 208-byte fixed portion, followed by a variable portion made up of a list of optional parameters. Each parameter has the following format: parameter identifier (PARM-ID) and parameter value (PARM-VALUE). The PARM-ID is a predefined BIN(2) value that identifies the parameter. The PARM-VALUE can be of any structural format as defined by the PARM-ID. If the parameter is not used it must be set to its minus value. For example, a PARM-ID of value -59 indicates that the Commitment control parameter is not active and should be skipped. If the parameter is used, its positive value is set in the PARM-ID field and the pertinent information associated with the parameter must also be set; by setting the PARM-ID value to minus, the system knows the length of the structure that will follow and how many bytes to skip. A special PARM-ID, 32767 (or hex 7FFF), indicates the end of the parameter list. Note that although the length of the fixed portion of the UFCB would not likely change across releases, the formats of the UFCB at different releases might be different slightly.

For documentation on the UFCB, you can refer to the following resources:

l Chapter 12, Input/Output in MI Using the SEPT, Chapter 13, File Conversion and Hashing, and Chapter 16, User-Defined 5250 Datastream I/O, of Leif Svalgaard's e-book discuss the UFCB in detail. Chapter 12 is also available in a presentation handout (handout.doc or handout.pdf) of the e-book for COMMON.

l Another useful reference is the Query (QQQQRY) API's documentation in the Information Center. It covers the parts of the fixed portion and the variable portion of the UFCB. Similar information can be found in source member QSYSINC/H.QQQQRY, which was shipped with the System Openness Includes option of i5/OS.

l Among the posts in the MI400 mailing list dating back to the early 2000s, you can find a lot of MI source examples and discussions about doing file I/O via the UFCB. Just go to http://archive.midrange.com/mi400/index.htm and search with keyword UFCB.

Perform or Disable Level-Checking via UFCB Parameters

Programs use two UFCB parameters to control level-checking when opening a file: the Level-check Option (lvlchk) parameter whose PARM-ID is 6, and the Record Formats (rcdfmts) parameter whose PARM-ID is 7. Formats of these UFCB parameters are shown in the following tables.

Maxnum indicates the number of array elements in formats. It tells the system the size of storage reserved for the rcdfmts parameter. The value of curnum ranges from zero to the value of maxnum. The system checks the first curnum record format name/ID pairs one by one against the file to open at runtime. For each record format name/ID pair in formats, if the record format name is not found in the file or if the RCDFMT-ID of the record format does not match that of the record format found in the file, the system raise a CPF4131 error message.

So, to tell the system to perform level-checking, a program should do two things. It should specify the lvlchk parameter and set the lvlonoff to 1 (i.e., set the PARM-VALUE of lvlchk to hex 80), and it should specify the rcdfmts parameter, set maxnum to the number of array elements in formats, set curnum to the number of record formats to check, and set the value of each record format name/ID pair.

To tell the system not to perform level-checking, a program should specify the lvlchk parameter and set the lvlonoff to 0 (i.e., set the PARM-VALUE of lvlchk to hex 00).

The following is an example MI program that performs or disables level-checking via UFCB parameters lvlchk and rcdfmts, lvlchk01.emi. It opens a file that has a single record format for read and then closes it via Data Management APIs QDMCOPEN and QDMCLOSE, respectively. LVLCHK01 takes two to four parameters. The first parameter is the target file to open. The second parameter of this program tells it whether or not to perform level-checking when opening the file (set it to hex 00 to disable level-checking, or set it to any other value to perform level-checking). If the second parameter is not hex 00, then the third and fourth parameters are expected to be the name and ID of the record format supplied for level-checking.

Compile MI program LVLCHK01 from lvlchk01.emi (via an MI compiler, e.g., mic) and test it.

Suppose that you have a database file called A5, which has a single record format called REC, the RCDFMT-ID of which is 28B9A5DECA650. You can do the following tests on A5:

1. Open A5 without level-checking: CALL LVLCHK01 (A5 X'00'). To make sure A5 is being opened, start a debug session before calling LVLCHK01 and add a breakpoint at LOOK using the following CL commands:

STRDBG LVLCHK01

ADDBKP STMT(LOOK)

After LVLCHK01 stops at breakpoint LOOK, press F10 and type DSPJOB OPTION(*OPNF) and find file A5 in the Display Open Files screen.

2. Open A5 with level-checking and supply the correct record format name/ID: CALL LVLCHK01 (A5 X'01' REC '28B9A5DECA650').

3. Open A5 with level-checking and supply the incorrect record format ID: CALL LVLCHK01 (A5 X'01' REC '28B9A5DECA650'). Now you'll get a CPF4131 error message.

Also it would be an interesting exercise to implement the above shown MI program in an HLL—for example, RPG or C. An RPG example, lvlchk02.rpgle, is available in Appendix A.

Achieve Compatibility Across Multiple Versions of a Database File by Using LVLCHK(*NO)

The method is quite straightforward:

• Never change field(s) in the existing record format of a database file (DBF).
• Always append newly added field(s) to the end of the existing record format of the DBF.
• Specify parameter LVLCHK(*NO) when creating/changing the DBF or overriding it via a file override command.

This way, an HLL program compiled to an older version of the DBF can happily work with all newer versions of it with no worry about the level-check errors. An HLL program that only writes to the DBF can even work with older versions of the DBF (of course, information kept in the fields not in the older version of DBF will be lost).

One application of this technique is the manner in which IBM deals with the output files of its numerous display commands whose OUTPUT parameter accepts the *OUTFILE special value, e.g., Display Object Description (DSPOBJD), Display Module (DSPMOD), and Display Journal (DSPJRN). Each of these display commands has one or more template output files in the QSYS library, corresponding to the record format(s) used by the display command. The template output files have no member and are created as LVLCHK(*NO). When a user requests a DSPXXX OUTPUT(*OUTFILE) OUTFILE(SOME_LIB/SOME_FILE) operation, the template file of the command is duplicated as SOME_LIB/SOME_FILE if it isn't already there. Certainly, a member will be added to the duplicated file, which inherits the LVLCHK(*NO) attribute from the template file. If you're interested in knowing the template file(s) of a given display command, the following post by Simon Coulter in the midrange-l mailing list might be helpful for you: http://archive.midrange.com/midrange-l/200105/msg01319.html.

The record formats of these output files are likely to change frequently from release to release due to newly added or improved functionalities of the system. But the changes to such a record format would only be new fields being appended to the end of it. For example, check the record format information of the template output file of the DSPOBJD command, QSYS/QADSPOBJ, at V5R2 and V5R4, respectively. You can get the following result.

Although, at V5R4, record format QLIDOBJD contains eight fields more than its V5R2 version, the beginning 104 fields of these two versions of record formats are exactly the same. So an HLL program compiled to the output file of a DSPOBJD command at V5R2 can work with an output file of a DSPOBJD command at V5R4 and vice versa. For the same reason, an HLL program compiled to the output file of a display command at one release can work with an output file of the display command at any release.

Appendix A: Source of ILE RPG Example lvlchk02.rpgle

Please refer to https://i5toolkit.svn.sourceforge.net/svnroot/i5toolkit/art/lvlchk02.rpgle for the most current version.

For more examples of using the MI instructions CALLPGMV and PCOPTR2, you can refer to System-builtin Headers and Examples for ILE HLLs. They are prototyped in mih-pgmexec.rpgleinc and mih-prcthd.rpgleinc, respectively.