In the past year, I was part of a team of Java programmers that transformed the way my company does business by completely rewriting its application using server-side Java. In this article, I offer some tips that have helped my team close the final gap from a system that almost does what you need to a fully functional, productive alternative to the green- screen world. Most of the lessons focus on the single biggest drawback to Java on the AS/400: poor performance. The opinions offered are my own and reflect my pragmatic view that systems development should be an evolution, not a revolution. You will have to judge for yourself if any of these suggestions are right for you. All the members of my company’s team of Java programmers developed the tips that follow.

Profiling Performance

In my article “Servlets: The New Application Architecture” on page 83, I give an overview of basic configuration of a server-side Java application. At the close of that article, I mention that an ill-tuned Java application can “run like a dog.” Take a look at some ways to make that application run like a racehorse. I think it is helpful to list the various pieces that have a part in the typical server-side Java transaction outlined in “Servlets: The New Application Architecture.” They are as follows:

• Network delays—When connecting over the Internet with a 56 KB modem, WebSphere may spend some time relaying your HTML output from the JavaServer Page (JSP) to the browser. I try to plan on a user being able to support a 3 kilobytes-per-second (KBps) transfer rate. So, if my resultant HTML file is 30 KB, I can expect WebSphere to take 10 seconds just to transmit the file from the AS/400 to the user’s browser. Keep the HTML simple, clean, and attractive and avoid animation or large images.

• WebSphere delays—When WebSphere gets a request from the browser, it has to recognize that it should run a servlet, start up the servlet, create a unique environment for the session, and pass the request to the servlet. The biggest bottleneck is servlet startup. A decent-sized servlet may take several minutes to load the first time it is accessed. You can

eliminate most of this delay by configuring WebSphere to load your servlets when you start your Web server subsystem.

• Servlet delays—Here is where your traditional programming skills come into play. Just as a poorly written RPG program is slow, a poorly written Java program will be slow.

• Database access delays—Getting data to and from the database on the DASD takes time. As I will discuss later, database I/O is the single worst bottleneck I have seen. Keep database I/O to an absolute minimum. You may even want to denormalize (gasp!) the database to help out this critical section.

• Environment delays—In the RPG world, you typically manage the environment with subsystems, priorities, classes, time slices, and memory pools. WebSphere runs on the AS/400, so it too can be managed with these techniques. A complicating factor, however, is the fact that the Java servlets run inside what is called a Java Virtual Machine (JVM). Think of the JVM as a layer between your Java servlet and the operating system. The JVM manages its own memory and has a number of properties you can set to affect performance. I will look at some of this later.

For the user’s experience to be as fast as possible, you first have to determine how the AS/400’s time is being spent. This means you have to profile the transaction and allocate a portion of the time to each of the categories that I listed. Start by picking a time when you have the AS/400 pretty much to yourself. You want a level playing field without any outside influences. Run the transaction from start to finish several times and record the total elapsed time from the time you click on the browser to when the results are finally displayed on the browser. This is the overall perceived response time. The goal is to make this as short as possible.

Next, take the resultant HTML that was displayed on the browser and save it to the hard disk of your PC. If you are using Microsoft Internet Explorer, click on the File menu and select Save As. Set the Save as type to “Web page, HTML only.” After you have saved the file to your PC hard disk, take a look at the size of the file. Assuming the HTML doesn’t include any references to image files (.gif or .jpeg), the HTML’s file size reflects how many bytes the JSP sent to the browser. Again, assume the speed will be about 3 KBps over the Internet for dial-up modems and divide the file size by 3 KB to estimate the network delay.

Nailing down the WebSphere delay is a little trickier. If the servlet is already loaded, it typically takes WebSphere less than 100 milliseconds to fire off a thread and create the session environment. To nail this down more exactly, end your HTTP server instance and restart it specifying this verbose option:

STRTCPSVR SERVER(*HTTP) HTTPSVR(myinstance ‘-vv’).

As the server starts up, a spool file named QPZHBTRC will be created for user QTMHHTTP. You can look at the time stamps in this spool file to see exactly how long it took for WebSphere to get a request to run a servlet, find that servlet, load the servlet (if not yet loaded), and then pass the request information into the servlet. You can also see how many bytes were in the response sent back to the browser. To end the traces, you will have to end the HTTP server and restart it normally (without the -vv option). Again, this delay is not usually a problem. If you see delays of over 100 milliseconds on a servlet that is already loaded, contact IBM and get some help.

Servlet delays and database delays are best quantified by writing some time stamps to the WebSphere log file. All servlets that are subclassed from javax.servlet.http.HttpServlet can call the log(String) method and pass a string to be written to the log file. Your WebSphere configuration controls the name and location of this log

file. The log file is just a normal ASCII text file, so you can look at it with Windows Notepad if you have mapped a drive to your AS/400. You can also use the ADMIN Web site on your AS/400 (by default, it is http://www.myinstance.com:9090) to display the log file inside the browser. Your time stamps in the log file will tell you how long the servlet took to run once WebSphere started it up. Of course, this time would include your database delays for any needed database I/O. Make sure your time stamps break out the database I/O by writing entries to the log file specifying something like “now accessing database...” and “database access completed.” If your situation is like mine, database I/O will end up dominating your quest to improve performance.

Environment delays are very hard to quantify. The most likely cause I have seen is initial heap sizes being set too small. This causes lots of extra garbage collection every time you hit the heap size limit and the JVM increases the heap (up to the maximum heap size). These parameters are controlled by ncf.jvm.mx and ncf.jvm.ms in the jvm.properties file. Assuming you are running on an AS/400 dedicated to WebSphere, you want all the heap you can get; so set the minimum heap size to be the same as the maximum heap size. I have heard general guidelines that the maximum heap size should be no more than half the total main storage of the AS/400. You may have to play with that upper limit to see what works for you. Messing with these parameters is not for the faint of heart. Make sure you record what any of the original values were so you can recover if you make a bad choice. You will have to end and restart the HTTP server for these changes to take effect. Again, I wouldn’t mess with the jvm.properties file unless all other options have been exhausted.

Performance Tips

So far, you have profiled the transaction, and you know where the time is being spent. Now you have to do something about it. This section explores the tips that have helped me most.

Record-level I/O Is Usually Faster than SQL I/O

If your performance profile is anything like mine, you will find that database I/O is the big performance killer in WebSphere. In the best of circumstances, I have never seen the kind of I/O performance in Java that I have come to take for granted in RPG, so anything you can do to help here has a big payback. Getting SQL to run fast on an AS/400 is an art, not a science. The bottom line is that you have to make the SQL optimizer happy every time. You may think you understand your SQL statement and even think the optimizer will use the obvious index already built over the table in question. However, I have learned from experience that the optimizer does not always make the obvious choices. Sometimes, you may have so many indexes built over a table that the optimizer times out trying to pick the best approach. Then, it may actually use the arrival sequence and build an index on the fly. If you are lucky enough to find an SQL statement that makes the optimizer happy, you can “freeze” the optimization by putting the SQL statement in a stored procedure or a prepared statement. Then, the optimizer will not try to reoptimize the SQL each time the procedure is executed. If you use dynamic SQL, the optimizer will rerun every single time you execute the SQL. I have run internal benchmarks of dynamic SQL, stored procedures with parameterized SQL, and other SQL techniques, and they usually don’t come close to record-level I/O using the IBM Java Toolbox for the AS/400. It is not uncommon to see performance increase by a factor of 12 for record-level I/O over SQL.

I get a big payback using record-level I/O when I can make my initial record selection all from one table—that is, all my initial selection fields are in a single table. I can then retrieve the records of interest. Once I have these records, I can selectively access other tables for supporting data. To do this, you may have to denormalize your tables somewhat. For example, if you really must select some records from the ORDER file within a given order date range and with customer names starting with SM, you may have to put both the order date and customer name in the ORDER file (even if the customer name

is also in the CUSTOMER file). Record-level I/O using the toolbox allows you almost surgical control over database I/O; you can reduce it to the bare minimum.

On the other hand, there seems to be almost no difference between SQL and record- level I/O on small tables where you can count on the optimizer to pick the right indexes. For example, if you have a state file with only 50 records keyed by state code, SQL can find the record for Tennessee just as fast as record-level I/O can.

Use the IBM Connection Manager if Using JDBC

Starting up and closing connections to the database is very costly. The best way to avoid this delay is to let the WebSphere IBM Connection Manager pool the database connections, keeping them open and ready for reuse. Using the Connection Manager is a little beyond the scope of this article, but it is not as hard as you might think. (See Sonjaya Tandon’s article on connection pools, “Pooling Party,” in the October 1999 issue of MC.) Using a connection pool is not much more trouble than managing the connection yourself. The big payback from connection pools comes when your servlets release Java Database Connectivity (JDBC) connections after each use. The Connection Manager will keep the connection open and put it in a pool where this servlet and any other can reuse it.

Here are a few words of caution: Always construct your code so that the Statement object’s close method will always be called. The Connection Manager will not release the SQL handle resources for you. If you forget to call the Statement close method, you will slowly use up all the SQL handles available on the system. Eventually, SQL will just stop working altogether, and you will have to restart the HTTP server.

Load All Servlets at Startup

By default, WebSphere loads servlets only when they are requested. This means the first user to access a given servlet is going to pay a big performance penalty while he waits (up to several minutes) for the servlet to load. You can easily use WebSphere’s administration facility to flag servlets to load at startup. It will take longer for the HTTP server to start up, but your users won’t see this long delay per servlet.

Use Data Pools for Small Tables

Data pools are an AS/400 feature that has been around for years and are not limited to WebSphere. You would be surprised by how many AS/400 shops aren’t aware of this nifty technique. You can basically force some tables or even just indexes to stay resident in main storage all the time. There, they can be accessed at memory speeds instead of DASD speeds. Details on how to configure data pools are beyond the scope of the article, but, in general, I like to set them up in a special subsystem I create called DATAPOOL. Configure the subsystem to have a private pool big enough to hold the files or indexes you want in main storage. You add an autostart job entry to the subsystem to run a little CL program. In the CL program, you put something like this:

CLRPOOL POOL(DATAPOOL 1)
/* load a whole physical file */
SETOBJACC OBJ(MYLIB/STATE) OBJTYPE(*FILE) +

POOL(DATAPOOL 1) MBR(*first) MBRDATA(*BOTH)
/* load just the keys from a logical file */
SETOBJACC OBJ(MYLIB/ORDERL1) OBJTYPE(*FILE) +

POOL(DATAPOOL 1) MBRDATA(*ACCPTH)

Keep JSP Output Small

As I have shown, it can take some time to send an HTML file across the Internet to a user that is on a dial-up connection. Take a look at the HTML content sent back to the browser. You may be able to use Cascading Style Sheets (CSS) or other techniques to reduce the

amount of repeated information sent back to the browser. For example, you can set the default font and font size for the whole page one time instead of constantly setting it in every paragraph. The opportunity is especially great for repeated elements such as entries in a table or list. I was able to reduce the HTML size of one of my company’s typical pages from 600 KB to about 40 KB. You can also easily detect the user’s browser type from within the JSP. That gives you the opportunity to take advantage of features unique to Netscape or Internet Explorer to further reduce the HTML content.

Convert Database from EBCDIC to Unicode

If you change the coded character set ID (CCSID) of all character fields in your database to 13488, then your Java code will not have to spend time converting all strings from EBCDIC to Unicode and vice versa. My company’s team noted an overall performance improvement of about 15 percent with this one simple change. Be aware, though, that this change will require you to modify all other non-Java programs that access these tables. For example, since Unicode uses two bytes for every character, your temporary variables in RPG programs need to be twice as big. You also will see a corresponding increase in DASD use. If you are desperate for performance, 15 percent is nothing to overlook.

Avoid Servlet-to-servlet Calls

Having one servlet call another often is expensive, just as calling one RPG program from another is time-consuming. You have to decide where to draw the line, but try to make sure that any such calls make sense to your servlet architecture and that it is never done in a loop. If you find that one servlet must make repeated calls to another servlet, reevaluate your design.

Miscellany

There are several tips I should mention that don’t deserve a separate category of their own. For instance, make sure you invoke the Create Java Program (CRTJVAPGM) command over all your servlets specifying optimization level 40. Also, make sure that all .jar files in your class path are also compiled to optimization level 40. You will have to check this every time you load PTFs that might replace any of the .jar files (as when you get that new PTF for the IBM Java Toolbox). You can optimize IBM’s Java Toolbox to level 40 with the following compile command:

CRTJVAPGM CLSF(‘/QIBM/ProdData/HTTP/Public/jt400/lib/jt400.jar’) +

OPTIMIZE(40) LICOPT(NOPRERESOLVEEXTREF)

Another tip is to use the StringBuffer class instead of the String class everywhere you can. I know this is common sense, but you would be surprised how easy it is to forget. Also, create classes as final and static wherever possible. A class where all the methods and class variables are final and static can be used without creating an instance reference, just as you can use System.out.println(String) without first creating an object of type System. The downside, of course, is that such methods can not be overloaded by future subclasses extending your classes.

Finally, if you are using the toolbox record I/O classes, make sure your AS400 class object specifies the QTMHHTTP user profile when connecting. If you don’t, the toolbox will not use the native access methods IBM specifically added to help toolbox performance; you will spend unnecessary time routing the data I/O through the TCP/IP stack.

Final Word

The tips mentioned in this article have enabled my company to radically improve servlet performance on the AS/400. The programming team still cannot approach the performance

of RPG programs and green-screens, but it is able to create workable solutions. IBM is constantly improving the speed of the components, so the situation will only improve over time. Space limitations keep me from going into much depth on any one topic, so use these guidelines to go off and run your own benchmarks on your own servlets. The benefits are definitely worth the trouble.

References and Related Materials

• AS/400 Performance Capabilities Reference—Version 4, Release 4 (SC41-0607-02, CDROM AS4PPCP2)
• “Pooling Party,” Sonjaya Tandon, MC, October 1999