Benefits that go beyond saving disk storage.

by Craig Pelkie

For most system designers, deciding how to store settings for program options is very simple: allocate a byte for each option. This method is simple to define and use, and the design proceeds with no further concern for that data element. But there are times when a more complicated definition would provide benefits beyond the mere storage of data. This more flexible alternative is to store the option as a bit within an option byte.

A project that I am currently working on involved this choice between two methods. I naturally chose the simple method first, in my haste to get on with the work, but was finally forced to reexamine and recode the data. The data that I needed to encode was a series of yes/no responses, representing both system options and user preferences.

Things Change...

The problem with system design (well, mine at least) is that it is impossible to analyze and predict all of the options that might be required or desirable. I started the project with the intention that the "meaningful" decisions in the system could be controlled by the client, rather than hard- coding in my own preferences. Although my clients were pleased that many options were available, they were quick to point out that other parts of the system should also be under their control. Also, each client had his or her own ideas about options that they wanted to control, so the additions were growing with each new client.

Traditionally, these types of requests could be easily satisfied by using part of the "filler" space that is usually included in a file. For the System/36 version of the product, this was no real problem, since I had included filler in the files to begin with. For the S/38 and AS/400 versions, the solution was a bit more complicated, since I don't include fillers, but add to the file as needed. This necessitated recreating the file, recompiling all of the programs using the files, and remembering to update the client version of the files. After this happened several times, I desperately began to wish for an easier way to accommodate the requests for additional options.

The solution that I found was to encode the options as bit fields. This involves more programming than the method of encoding one byte per option, and there are some other tradeoffs that are made. Overall, the method has worked well and lets me freely add options without any of the problems I described. I'll tell you the mechanics of the technique and review some of the concerns with you.

A Single Character

A "bit field" is really nothing more than a one-byte character field. The difference is that programs using this one-byte field treat it not as a byte, but more like a miniature array of eight bits. Each bit within the field can have a value of 0 or 1, and you can assign whatever meaning you want to those values. Usually, you would think of those as being "yes" or "no" responses. For example, you might want to allow your users the option of answering a confirm message before updating a file. I also use the values for other "binary" situations: for example, 0 indicating a monochrome monitor, 1 indicating color (which I suppose is really another type of yes/no setting).

The point is, a single bit can represent two settings. In some cases, you might need more than two settings. You can still use bit settings. For example, using two bits allows you to represent four choices (with settings of 00, 01, 10 and 11). Three bits allow up to eight settings; the number of choices equals 2 to the Nth power, where N is the number of bits you're using. On the other hand, there are instances where it is easier to simply define a separate option field and store the setting as a separate byte. I don't try to force everything into a bit field, especially when it's obvious that it's not the best solution.

Getting Started

The first decision you'll have to make when using a bit field is what the purposes of the assignments that you're making are. Each bit field contains eight bits, numbered (from left to right) 0 to 7. Why the numbering within a high level language starts at 0 is beyond me, but it's important to know that the compiler expects us to abide by that convention. You must decide on the meaning of each bit. For example, you might decide that bit 0 means that the user wants to have the cursor set to blinking or no blink, and bit 1 means they want the alarm to sound or remain silent when they get an error message. The point is, you decide this yes/no stuff before doing anything else, and you'd better write it down in an obvious place, since you'll be referring to it frequently. On the AS/400, I favor including the bit settings as comments within the DDS. For the S/36, I have all of my file definitions in /COPY modules, and include the comments there as well. I think it makes sense to include the comments within the file definition, since the file can be referenced from many programs. If you put the comments in a program, then you must always remember which program includes the comments.

Once you've decided on the meaning for a particular bit setting, commented it, and resolved to never use that position for anything else, you're almost ready to start using it. Naturally, you'll have to first define the bit field itself in your files. This is as simple as defining a one-position character field. In my project, I have more than eight options that I'm setting, so I defined multiple bit fields. In fact, I have even included a few "filler" bit fields that I'm not using yet, so I can avoid the problem of having to redefine the file later. I have included 10 bit fields in my file definition, which allows me to define up to 80 options. Although I don't feel that saving space is as big an issue now as in the past, you can see that I've saved quite a bit of space by using bit fields.

One of the problems that you must be aware of concerns the default initialization of the bit fields. That is, when your program writes your new record to the file, the character fields are set to blanks unless you set them otherwise. "Blank" is x'40', not x'00', meaning that bit 1 of your field is set ON. You should be careful to initialize bit fields, either to the settings that you want or to x'00', before using them.

Setting and Testing

Longtime readers of Midrange Computing are probably familiar at least with the techniques of setting and testing bit fields, since there have been many articles dealing with this subject. The RPG operation codes are not particularly difficult to understand, and there are equivalent operations in the other high-level languages (although I can't think of any technique to use bit-level operations conveniently in CL programs). The RPG opcodes you'll use are BITOF, BITON and TESTB. For this article, I'll point out only what I've used to accomplish my purposes, and avoid discussing some of the exotica of the opcodes, particularly TESTB.

1 shows an example of how you can initialize a bit field and set options within the field. The bit field SYSOPT is first set to x'00'. Various user preferences are then set; for example, cursor blinking is set in bit 0, alarm sounding in bit 1, and confirm messages in bit 2. If you have additional options, you can continue them in bits 3 through 7, or (this was the important part for me) leave bits 3 through 7 "reserved for future use." For me, that was the reason to convert from the "one byte per option" method to using bit fields. In the future, when a client requests another option, I already have five bits available. I can choose any one of those five bits to store the option setting, without altering anything else in the file. Since the byte is already in the file, I don't need to be concerned with adding another option byte to the file, with the attendant headaches of changing the record structure.

Figure 1 shows an example of how you can initialize a bit field and set options within the field. The bit field SYSOPT is first set to x'00'. Various user preferences are then set; for example, cursor blinking is set in bit 0, alarm sounding in bit 1, and confirm messages in bit 2. If you have additional options, you can continue them in bits 3 through 7, or (this was the important part for me) leave bits 3 through 7 "reserved for future use." For me, that was the reason to convert from the "one byte per option" method to using bit fields. In the future, when a client requests another option, I already have five bits available. I can choose any one of those five bits to store the option setting, without altering anything else in the file. Since the byte is already in the file, I don't need to be concerned with adding another option byte to the file, with the attendant headaches of changing the record structure.

Setting options into bit fields isn't much use unless you can later decipher the settings, and 2 shows you the coding you would use to determine an option setting. In this example, we want to determine whether the alarm should sound when we display a message. Since this option was originally stored in bit 1 of the option byte, we test that bit to determine whether it is "on". Using the TESTB opcode with an indicator in the "equals" position sets the indicator on if the bit is on, off if the bit is off.

Setting options into bit fields isn't much use unless you can later decipher the settings, and Figure 2 shows you the coding you would use to determine an option setting. In this example, we want to determine whether the alarm should sound when we display a message. Since this option was originally stored in bit 1 of the option byte, we test that bit to determine whether it is "on". Using the TESTB opcode with an indicator in the "equals" position sets the indicator on if the bit is on, off if the bit is off.

You can review the RPG manual for each of these opcodes, and learn about the additional combinations of settings and indicators that you can use. I have only shown here the essentials of setting and testing bit fields used for options. If this is your first experience with bit fields, I suggest that you stick with simple usages like those shown until you become more familiar with the techniques. On the other hand, this is about as complicated as it gets, so there's really not a lot more to learn at this point.

Pro and Con

Deciding whether or not to use bit fields is not really a point of debate, say, along the order of not using GOTO/TAG in programs. It is more a matter of planning, deliberation and weighing the consequences.

On the "pro" side, I have found that setting up my 10-bit fields in the file has let me define all of my current options, and has left me plenty of room for future needs. I've tended to group similar types of settings into the same byte field; for example, terminal control options into one byte, options for a particular program into another, and so on. This means that each bit field usually has one or two unassigned bit settings, which is exactly what I want. I want it that way so that in the future, when another option needs to be set, I can set it in the byte that makes the most sense.

By using bit fields, I am able to assign a great number of options in a small amount of file space. Although I'm not usually frantic about saving a few bytes in a control or options file, I find it a comfort to be able to have a few option bytes available, already assigned, knowing that I can get eight times the work out of each byte. By making each byte pull more weight, I don't have to worry about going back into the file definition, adding another byte field, then recompiling and updating my clients' files again.

On the other hand, you have probably already seen several potential problems with bit fields. First, you must quite carefully map out the bit assignments. In a single byte option field, this would not be such a concern, but you don't want to mix up your assignments among bits. Second, you should probably take care to initialize all of the bits in the bit field. This usually amounts to setting all of the bits OFF, as shown in 1, then setting bits ON as needed.

On the other hand, you have probably already seen several potential problems with bit fields. First, you must quite carefully map out the bit assignments. In a single byte option field, this would not be such a concern, but you don't want to mix up your assignments among bits. Second, you should probably take care to initialize all of the bits in the bit field. This usually amounts to setting all of the bits OFF, as shown in Figure 1, then setting bits ON as needed.

The programming required to set and test bit fields is also more involved than using single byte fields. For example, rather than just move a "yes" or "no" value into the setting for the "blink cursor" option, you must now use the less obvious BITON opcode. This also can be a problem when you use the setting; for example, without proper documentation, it would be difficult to discern the meaning of the bit test in 2.

The programming required to set and test bit fields is also more involved than using single byte fields. For example, rather than just move a "yes" or "no" value into the setting for the "blink cursor" option, you must now use the less obvious BITON opcode. This also can be a problem when you use the setting; for example, without proper documentation, it would be difficult to discern the meaning of the bit test in Figure 2.

Another problem that you must think about before using bit fields is retrieving the settings, other than in programs. Will you need to test the setting of an option for a sort or a logical file create? As an example, you might think that you could code a gender setting of male/female as a bit setting. But before you do, you should consider whether you would ever use that value outside a program, where there are opcodes to test the setting. Chances are, something like a male/female option should probably be encoded as a separate byte.

So what can, or should, you encode as a bit field? It's hard to say, but in my experience, I have found that many system options and user preferences can be encoded. It is unlikely that those settings will ever need to be referenced outside of the programs that use the settings, so I don't have the sort/logical file concern. It's tough to say that experience will teach you, but I've occasionally had to move a bit setting back to a byte field. However, at this point, the pure bit settings are more obvious, and when I recognize that an option must be a byte field, I don't try to force it into a bit field.

If you haven't seen or used this concept before, you may wish to consider it when you start your next project. I doubt if the technique would lend itself to "retrofitting" into existing code, since the programming involved, as shown in the two figures, can become quite a bother. But for new work, you might decide that the effort required for additional coding is offset by the advantage of a more stable file definition. Readers with experience or additional thoughts on this technique are welcome to share their opinions with us, so that this little known technique might become more widely known.

Using Bit Fields as Data

Figure 1 Initial bit settings

 Figure 1: Initial Bit Settings .20....+...30....+...40....+...50 BITOF'01234567'SYSOPT BLINK IFEQ 'Y' BITON'0' SYSOPT END ALARM IFEQ 'Y' BITON'1' SYSOPT END CONFRM IFEQ 'Y' BITON'2' SYSOPT END 
 
 Using Bit Fields as Data
 
 Figure 2 Testing a bit setting
 
 Figure 2: Testing a Bit Setting .20....+...30....+...40....+...50....+...60 TESTB'1' SYSOPT 90