Have you ever considered using the OS/400 Integrated File System stream files to store your data but didn’t do it because you wanted better data reliability? Have you desired the ability to obtain detailed information about changes made to files and directories under a specific file system’s directory tree? Would you like to recover your file system objects back to a state prior to when a third-party application corrupted it? Would you like to maintain a backup system containing your OS/400 IFS data so that it can take over if the primary system fails?

If you answered “yes” to any of these questions, you should consider taking advantage of the new support available in OS/400 V5R1M0 iSeries. This new support provides journaling of key OS/400 IFS objects.

Even if you answered “no” to all of the questions, you should still read on and learn about the benefits of journaling of OS/400 IFS objects. There are more features you may find useful in other system activities relating to OS/400 IFS objects.

What Is Journaling?

What does journaling really mean? Journaling helps you recover changes to OS/400 IFS objects that have occurred since your last complete save. It provides a way to track detailed information about object changes. Journaling also provides built-in protection against partial changes made to an object at the point of an abnormal system termination. In previous versions of OS/400, only database files and access paths could be protected with journaling.

You use a journal to define what objects you want to protect with journaling. This is often referred to as journaling the object. A journal receiver object contains the journal entries that the system adds when operations occur that are journaled, such as data or attribute changes to a stream file, or when renaming links within a directory.

Additionally, you can use the remote journal function to set up journals and journal receivers on a remote iSeries or AS/400 system running OS/400 V5R1M0. These journals and journal receivers are associated with journals and journal receivers on the source system. The remote journal function allows you to replicate journal entries from the source system to the remote system. This replication would help you maintain a copy of your file

system objects on a remote or backup system. Figure 1 shows an example of a simple replication environment.

Given that journaling is considered a building block for system-clustering technologies, the addition of this support in OS/400 IFS increases the realm of clustering support in OS/400. This support may be used by high-availability, business-partner applications or by your own applications to provide a more complete user solution. For more information on system clustering, check out the article “Clustering and the iSeries 400” (Steve Finnes and Mike Synder, MC, December 2000).

New in V5R1...OS/400 IFS Journaling!

The new journaling support for OS/400 IFS objects provides three primary functions. One of those functions is recording changes to objects. OS/400 IFS object changes are recorded in journal entries in the journal receiver. The entries may then be used for various reasons including the recovery discussed in the second and third primary function, which I will discuss later in this article.

Another use of the recorded changes is for replicating the entries to a backup system so that they can be replayed to create and maintain a replica of the source system. The replica could be used in the case of a source system failure, or it could prove useful in environments that have dedicated nonproduction systems for development and burn-in of new applications. It may also allow you to divide your workload between more than one system.

Journal entries can also be used to track activity on objects for either audit purposes or to assist in the development of software when testing or debugging. Remember those times when you were debugging a program problem and wished there was a way to determine if and when a specific change was occurring on one of your objects? This problem can be solved by simply starting journaling and looking at the journal entries for that object. Another possibility is to use the Receive Journal Entries (RCVJRNE) command. That command allows you to specify an exit program, a journal, and optionally, a list of journal entry subtypes. Whenever one of those entries is sent to the journal, the user’s exit program is asynchronously called with the journal entry.

Single-system Recovery

The Single-system recovery is the type of recovery that occurs during an IPL, following an abnormal system termination (i.e., “crash”). At the conclusion of the IPL, you can have confidence that all in-flight changes to your OS/400 IFS objects have been completed or backed out as a result of the journal recovery processing, which is an integral part of the IPL process. This can provide protection against objects damaged by system functions that were interrupted during their critical operations. Journal recovery processing also eliminates the need to restore the last saved version of an object and lose any recent changes. Possibly best of all, once your objects are journaled, this function of journaling is one from which you can benefit without changing any of your existing applications or writing new ones.

Recovery of a Saved Object to a Known State

This is typically an administrator-initiated recovery that provides you with a mechanism to recover a saved object to a specific state. This state may occur some time after the last save but prior to an operation or series of operations that caused the object to become corrupted. It’s also typically what ensues at a hot site when you’re attempting to recover from a site disaster (e.g., explosion, hurricane, fire). Database objects have had this hot site recovery support for years, and now OS/400 IFS objects do, too! Here’s how it works.

First, the administrator restores a recent copy of the object. Then the desired state is determined from the journal. The Apply Journaled Changes (APYJRNCHG) command is

then used to apply journal entries to the restored object up to the last “good” change before the first change that caused the corruption.

Having this ability also may reduce the save activity that you use to protect your objects. Instead of saving your objects nightly, you could save them weekly and then only save the journal receivers nightly. Since the receivers only contain changes in objects and not the entire objects themselves, less data will need to be saved.

This new journaling support is now available for objects of type stream file (*STMF), directory (*DIR), and symbolic link (*SYMLNK). These three object types can be created into and reside in any of the following file systems: the root (represented by a slash [/]), QOpenSys, and user-defined file systems.

What’s Getting Recorded?

Objects that are being journaled will have nearly every change recorded in the journal receiver. The inclusion of journaled directories provides enough recorded information to effectively replicate an entire directory tree structure to another system.

For example, saves, restores, and attribute changes of stream files, directories, and symbolic links are recorded. Security changes such as authority, auditing, and owner/group changes are also recorded. Object destroys are recorded for all three object types (i.e., stream files, directories, and symbolic links). Open and close operations may optionally be recorded for stream files and directories. In addition, the action of synchronizing a file’s data with its associated storage device will be journaled. The system also records data changes to stream files. These include data write, truncate, and extend operations.

Directories have several operations whose changes can be recorded. First, operations that create new objects and link them into a journaled directory will be recorded. These include operations that create directories (e.g., the mkdir API or CRTDIR command), create files (e.g., the creat API or COPY command), and create symbolic links
(e.g., the symlink API or ADDLNK command). Additional operations that cause new links to be added to a directory are the Add Hard Link functions (e.g., the link API or ADDLNK command). In order to provide a complete directory solution, you will also need to be aware of changes to these links. This is provided by journal entries for renaming, moving, or removing the link.

Journal entries can contain images that are referred to either as “before” images or “after” images. Before images contain, just as they imply, a view of the portion of the object that is being changed prior to the change. An after image contains the new information for the object after the change. The after images are always required for system recovery use, but the before images are optional. However, OS/400 IFS journaling currently only supports after images.

I have listed several interfaces that can cause journal entries to be created. However, this is not a complete list. Journaled OS/400 IFS object changes will be journaled regardless of their origin. Therefore, changes made from a PC client using Operations Navigator or a network drive, using the OS/400 Portable Application Solutions Environment (PASE), or using the Qshell interpreter will all be journaled.

Journal entries emanating from journaled objects residing in libraries are typically identified by library-qualified object names. In contrast, OS/400 IFS journal entries identify the associated journaled object through the file identifier (file ID). Every OS/400 IFS object can be uniquely identified by its file ID. The file ID can be passed into the Qp0lGetPathFromFileID API to convert the file ID associated with the journal entry into a pathname to the object. However, many of the directory operations’ journal entries contain a National Language Support (NLS)-enabled path name within them in addition to the file ID. This NLS-enabled path name in the entry is in the same format used by the NLS- enabled OS/400 IFS APIs. Using these APIs, which are new in V5R1M0, eliminates the need for copying and converting path name characters.

Features, Features, Features

The new OS/400 IFS journaling support also has some special features that make the user’s role in journaling much easier. In order to maintain an entire hierarchical directory tree structure under journaling, the user, or the user’s application, would have to continuously monitor for new objects being created and linked within the structure. Then, the user would need to establish journaling for that object. This leaves a potential timing window in which changes might be made to the object before the start journaling operation has completed. Those changes would not have been recorded.

In order to avoid the risk of changes not being recorded, OS/400 IFS journaled directories support an option that eliminates the timing window. When journaling is initially started on a directory, the user can specify the Inherit Directory’s Journaling Status attribute. When this attribute is specified, each object that is linked into the directory will have journaling automatically started. The implicit start journaling operation will use the same journal and options as the directory in which it is being linked.

While that feature provides for easier maintenance on the tree structure, there is another special feature that assists in the initial journaling setup. Both the API and command interfaces that are used to start journaling on an OS/400 IFS object have an option described as subtree processing. When this option is requested and the object to be journaled is a directory, all objects within that directory’s subtree are also processed. This is a nice feature to take advantage of when deciding to protect an existing directory tree structure with journaling.

Setting It Up

Though there is some planning involved when considering journaling of OS/400 IFS objects, the actual process of setting up journaling is rather straightforward. First, you create a journal receiver and then attach the journal receiver to a journal on a create journal command. The start journaling operation can be done from either a CL command (STRJRN) or an API (QjoStartJournal). Both interfaces provide the same set of functionality and options. These interfaces are pathname-based for both the object and journal parameters. You may specify up to 300 individual path names, but that doesn’t limit you to start journaling on only 300 objects per call. You can use the wildcard characters asterisk (*) or question mark (?) in the last component of each path name to cause this list to grow far beyond 300. Additionally, another option provides the ability to have the system traverse the entire subtree of each directory specified in the list of path names. Combining these options, the user is limited only by the maximum number of objects that may be journaled to a single journal, which is currently 250,000.

If you are already journaling database files or access paths, you can journal your OS/400 IFS objects to that same journal, if desired.

Additionally, you can also journal data area and data queue objects in OS/400 V5R1M0, and these object types may be journaled as well to the same journal as OS/400

IFS objects and existing database objects.

There are several other CL commands and APIs that are new or have been enhanced to include the above OS/400 IFS object-related parameters, incorporating the same options. The opposite of the start journaling operation is to end journaling. These interfaces are the new CL command (ENDJRN) and the API (QjoEndJournal). Also, the same CL commands and APIs that can be employed to manage and view journaled database objects can generally be used in a similar fashion to deal with journaled attributes of OS/400 IFS objects. The exception is that commitment control is not specifically supported for IFS OS/400 objects.

To facilitate the application programs that will be managing the start and end of journaling across multiple systems, several of these interfaces also provide the ability to identify journaled objects by their file ID. An object’s file ID can be obtained from the Get

Attributes API (Qp0lGetAttr), Work with Object Links (WRKLNK) attribute display, or from journal entries themselves. Applications in a high-availability environment driven by journal entries must also consider how to match OS/400 IFS objects on a production system with their corresponding replicas on a backup system. Since file IDs are recorded in the journal entries generated on the production system, a replication application could map them to replacement file IDs on the replica system by means of a file ID mapping table. Such a table might include the path names on both the source and replica systems. You should be aware that file IDs change during object restores, and that the journal support attempts to help you compensate for this by providing file ID mappings within the journal entry for the object restore.

Want to limit your green-screen use? Operations Navigator has been extended to provide you with GUI support for starting journaling, ending journaling, and displaying an OS/400 IFS object’s journaling status, all from your PC interfaces.

Figure 2 (page 87) shows the dialog box that is brought up when accessing the Journaling menu item from either the File menu or the pop-up menu. In this example, the directory named Company_Data is not yet journaled, and as the Status indicates, it has never been journaled. This start request will include all objects within the directory’s subtree and will set the Inherit Directory’s Journaling Status attribute.

Good News!

Now that you have learned about OS/400 IFS journaling, what will you be saying to your users about their data when your system crashes in the middle of a busy day? If you have employed OS/400 IFS journaling in your operations, it will be good news for your users.

REFERENCES AND RELATED MATERIALS

• AS/400 Clustering Technology white paper: www-1.ibm.com/servers/eserver/ iSeries/whpapr/cluster.htm
• iSeries Online Library: http://publib.boulder.ibm.com/pubs/html/as400/online/ homeeng1.htm (This library contains information on backup and recovery as well as information on the AS/400 IFS, CL Reference, and System API Reference.)

JLB1/RCV1

Local Journal Local Receiver Remote Receiver Remote Journal

Figure 1: This is a high-level view of a simple replication environment.

JLB1/JRN1
PRIMARY SYSTEM BACKUP SYSTEM

JLB2/RCV2 JLB2/JRN2

Directory tree journaled to JRN1 on primary system

Directory tree replica of primary system’s tree

Replicator application to receive and replay journal entries

Figure 2: This dialog window is used to start journaling from Operations Navigator.

Also Read

Practical Middleware: Architecting for the Future

Practical Middleware: Starting the Journey

IBM i 7.2: The Carrot Versus the Stick