In my last TechTip, I provided a U.S. ZIP code search program that uses AJAX to display potential results as you type. I mentioned that this technique would cause a lot of extra database access and that one way to avoid a lot of open/close of the database could be to use a data queue to communicate between the RPG CGI program and the database. Now, I have created a programming model that shows how to do just that. (Sorry! Those cool HTML/JavaScript features that I promised to write about will have to wait until the next tip.)

Before I start, let me explain why this is an issue at all: Every time an RPG CGI program is called, it's called through the Web server's CGI job. The Web server watches port 80, waiting for something to happen. When a CGI request arrives, it executes the request, calls a CGI program, opens the files, reads the records, closes the files, and passes data back to the browser. It then eagerly watches port 80 again, ready to answer more requests. If we can minimize the open/close of the database files—or even better, keep the database files open—we can gain a performance advantage. And that's what this TechTip is about. Figure 1 shows how the programming model works.

Figure 1: For maximum efficiency, keep your database files open. (Click images to enlarge.)

A: The browser makes a request and waits for the Web server to answer.
B: The Web server "catches" the request and executes the RPG CGI program.
C: The RPG CGI program places an entry in the data queue INBOX at D and then waits for the answer to appear in the REPLY data queue at F.
E: A never-ending CGI server program waits for something to appear in the data queue INBOX, and when it does, the CGI server program reads the data queue entry, processes it, and writes the answer back to the REPLY data queue.
C: The RPG CGI program reads the REPLY data queue, creates an HTML reply, and writes it back to the waiting browser.

Wow! This is a lot of waiting. Does this really work? Yes. And it works very fast.

But how does the RPG CGI program know which entry to read from the REPLY data queue? The keyed data queues are the nuts and bolts of all this. If you have not worked with keyed data queues before, here is what the help text on the CrtDtaQ command says: "Data queue entries are received by key. A key is a prefix added to an entry by its sender." So the challenge is to generate a unique key that can be passed around with the data entry and to make sure that it is picked up by the correct requester.

For that purpose, I use a Universally Unique Identifier (UUID), which is a 16-byte string that is guaranteed to be unique. My good friend Carsten Flensburg helped me write the GETUUID program used in this example, and just for fun I created a file and wrote 10,000,000 records into it using GETUUID to generate the keys. None came out the same, so let's assume that it works the way we want.

But what if I want to use the INBOX to serve various CGI server programs? The INBOX data queue is also keyed, and I just use the name of the program and library that are to read the data queue entry. This is a very simple way to solve this problem, and it ensure me that a lot of different programs can monitor the INBOX data queue without "stealing" something that does not belong to them.

I have created a small library called CGISERVER, which contains the data queues, the data files, and the RPG programs.

Here is an overview of the functions of the programs:

To install the CGISERVER library and test it, do the following:

1. Download the zipped save file cgiserver.zip.
2. Create a save file called CGISERVER somewhere on your i5.
   
3. Unzip and FTP the save file to your i5 Web server to the save file you just created.
   
4. Restore the CGISERVER library with the following command:
   RSTLIB SAVLIB(CGISERVER) DEV(*SAVF) SAVF(yourlib/CGISERVER)
   
5. Add CGISERVER to your library list.
6. Submit never-ending program SRV000 to a job queue that accepts more than one jobq entry (for example, QUSRNOMAX) with the following command:
   SBMJOB CMD(CALL PGM(SRV000)) JOB(SRV000) JOBQ(QUSRNOMAX)
   You might have to twist the command with a proper user profile. Also, remember to check whether QUSRNOMAX is active; alternatively, you can use QSYSNOMAX.Note: If you want to stop the SRV000 program, just call STP001, which will send a "*STOPPGM" entry to the server program.
7. To test, you can call program REQ001 by entering CALL REQ001. This will fetch 50 entries from database USZIPCODES and print a primitive list (nothing fancy but good enough to see that things are working).

When you look at the programs, you will see that I use QUALIFIED data structures to move the data around. This could be done using arrays or strings, but remember that this is just a model to show you how to remove databases from CGI programs and move them into never-ending CGI server programs.

Because I want to make this work in my U.S. ZIP code search program, I created a new RPG CGI program called FORM007, which replaces FORM005 that was used in my last tip.

To install the FORM007 RPG CGI program, do the following:

1. Download the source.
2. Unzip and FTP the source to your i5 CGI library and follow the instructions in the header section to compile it.
3. If you have installed code from the "Give Me That ZIP Code, Please" TechTip, you have to make a change in the JavaScript that handles the call to the Web server. In directory /ajax, locate the file called getzipcodeus.js and find the following line:
   var url="http://yourserver:81/cgi-bin/form005"
   Change it to
   var url="http://yourserver:81/cgi-bin/form007"
   
4. If you have not installed the code from "Give Me That ZIP Code, Please," you can just call FORM007 from your browser:
   
   http://yourserver:81/cgi-bin/form007?city=farmerville&zipc=&list=10
   

Be sure to have SRV000 running before you test. Otherwise, you will get the message shown in Figure 2.

Figure 2: Avoid this error message.

This completes this TechTip, which I hope can help you make your CGI programs perform a little better in a quite simple way.

But before I fade away, I have three more things to mention:

If necessary, it is very easy to scale the CGI server programs. Just call them again and you add some horsepower. Very easy.

1. If you have some data that resides on a different i5 box than your Web server is running on, just create a remote data queue and you are done. I did a test in our environment, and it took me no more than 10 minutes to set up a working example.
2. Data queues are a very cheap and simple way to send data back and forth quickly. If you need a more robust and secure way to send data, consider WebSphere MQ (formerly MQ Series).

Stay tuned! Cool HTML/JavaScript features are coming up in my next TechTip.

Jan Jorgensen is a programmer at Electrolux Laundry Systems Denmark. He works with stuff like RPG, HTML, JavaScript, and Perl. You can reach him at This email address is being protected from spambots. You need JavaScript enabled to view it..

Jan Jorgensen is one of the owners of www.reeft.dk, which specializes in mobile and i5 solutions. He works with RPG, HTML, JavaScript, Perl, and PHP. You can reach him at This email address is being protected from spambots. You need JavaScript enabled to view it.. 

 

Also Read

Programming in ILE RPG - Defining Data Structures, Part I

H and F Position-less Specs

Practical RPG: AI and IBM i