By now, almost everyone knows that Sweden’s Intentia International AB is the first big enterprise resource planning (ERP) software vendor to get a full Java suite out the door. The question on everyone’s mind is “How well do server-side Java applications perform?” “Write once, run anywhere” means little if Java applications can’t make efficient use of expensive server processing power.

The AS/400 has benefits when comparing legacy applications written in RPG and COBOL with new Java applications. For instance, Java applications run within a software abstraction layer known as a Java Virtual Machine (JVM). Similarly, RPG and COBOL applications are rather remote from the underlying processors in each AS/400. Both new Java applications and old RPG and COBOL applications are about equally far from the iron. In contrast, PC servers use C and C++ code, and mainframes use assembly language and COBOL programs that are written much closer to the iron. On a UNIX or mainframe server, a JVM is arguably further away from an underlying processor than it is on an AS/400, too. So when Java application benchmark tests become available for various midrange servers, comparisons between Java and legacy applications on UNIX and PC servers probably won’t be very encouraging at first, and ditto for mainframes. They will probably look encouraging for AS/400s, however.

Another major benefit the AS/400 has in Java performance is its single-level storage architecture. AS/400s do not distinguish between main memory and disk capacity, at least not at the application level; OS/400 keeps track of what needs to be in main memory and what can be swapped out to disk. With object-oriented applications built in Java, applets are loaded into memory and used and must then be removed from memory once they are no longer needed. This is called garbage collection. These garbage collection functions must be added to Windows NT, UNIX, and mainframe operating systems, and they eat up computing power. But OS/400 single level storage already functions as a garbage collectors.

However, the AS/400’s performance edge over other servers does not mean that Java is ready for prime time, because it isn’t. But it’s getting there much faster than any of us expected. That is why IBM is now developing a new Java server benchmark test, as yet unnamed, that it hopes will be able to show the AS/400 as the server of choice for Java applications. This benchmark, say insiders at IBM, will be much like the industry standard

TPC-C online transaction processing benchmark but will be written in Java instead of other languages. IBM plans to have preliminary test results for V4R3 servers soon and hopes to have V4R4 results by June. I’ll keep you posted on this Java benchmark as it develops.

For now, the only benchmark data worth discussing for Java server applications is the limited performance specifications comparing Intentia’s RPG with Java Movex ERP applications. To understand benchmark results, you need to know a little about the latest Movex suite, which Intentia delivered in January. Movex is a popular application suite among midrange companies in the food and beverage, fashion, steel, and paper industries. Movex 11, the latest version, comes in two releases, both of which run on OS/400 V4R3. The This Generation (ThisGen) release of Movex 11 is written in RPG (Original Program Model [OPM], not ILE), while the Next Generation (NextGen) release is written in server- side 100% Pure Java. Intentia did not use IBM’s SanFrancisco templates to lay the foundation for its version of Java but developed its own conversion tool to transform its highly structured RPG code into Java. Intentia says that its tool was able to convert about 92 percent of the 23 million lines of code in the Movex RPG suite to Java. After the other 8 percent of the code was tweaked by hand, the suite had only about 8 million lines of Java code. The reason for the drastic decrease in the number of lines of code is that the old RPG programs are monolithic. Consequently, every time the suite needed to perform a function, the lines for the function had to be added to the code.

Java, on the other hand, is an object-oriented programming language that allows the application to point to a single set of code (an object) to perform a specific function whenever needed. The upshot of Java is that it takes fewer lines of code to build a suite and it becomes easier to maintain the application because there’s only one place to look for particular objects. The Intentia tool, which has no name, works in both directions. All current and future Movex development will be done in Java, which will then be run through the tool to create OPM RPG programs. Intentia has no plans to work in ILE RPG and says it will keep the functionality of its ThisGen RPG and NextGen Java suites in synch for years.

Intentia says that both ThisGen and NextGen have been optimized for the AS/400. Both the RPG and Java server code in the different implementations of Movex 11 drive the same C++ client screens and are indistinguishable to end users. Initial benchmark tests on the beta implementations of Movex 11 show that the throughput of NextGen is not yet up to par with the ThisGen RPG implementation but is probably much closer than expected. Intentia recently tested both Movex ThisGen and NextGen on an AS/400e model S20-2166 server running OS/400 V4R3. The machine was equipped with four 100-MHz Apache processors, 4 MB of level 2 cache memory per processor, 1 GB of main memory, and 185 GB of disk capacity. The software ran in a two-tier implementation with the Movex database and application programs running on the same AS/400. (Most ERP vendors test their software in three-tier configurations, separating application logic from database serving; this allows a database server to do more work. IBM’s CPW tests are done in this manner.) On Intentia’s Movex tests, the S20 could process 596 orders per hour running ThisGen (RPG) with an average response time of 1.51 seconds and CPU utilization of 12 percent. That same machine could process only 201 orders per hour running NextGen (Java). In addition, response times increased to 2.84 seconds, and CPU utilization grew to 19 percent. This translates to about 80 Movex users for the RPG suite and 27 users for the Java setup on this machine.

With CPU usage so low on both the RPG and Java implementations, Intentia and IBM are obviously working to tune Movex to V4 software so it makes fuller use of processor resources to improve throughput. IBM is also adding a new JVM (level 1.1.7) to OS/400 V4R4, which will ship in May and which IBM expects to offer 25 to 50 percent better performance than the JVM in OS/400 V4R3 (level 1.1.6). Perhaps more significantly when it comes to performance (and neither IBM nor Intentia talk about this), every server platform is limited to a certain number of concurrent application threads that it can support, regardless of the underlying processing power of the machine on which the JVM is

running. Most JVMs can’t handle more than a dozen or so threads. With a Java 2 JVM (formerly level 1.2 announced last December) and improved Java Database Connectivity (JDBC) access, both the number of threads supported and application performance should increase perhaps as much as 40 to 50 percent again. (Both the Java 2 JVM and improved JDBC access could come to market either by the end of 1999 in a PTF for OS/400 V4R4 or by early 2000 with the Pulsar series of AS/400s and a new version of OS/400.) That should put precompiled Java application performance within 25 to 30 percent of the throughput of native RPG applications, which would be close enough for government and private sector work.

Domino Costs on the AS/400 Revisited

One of the problems of magazine publishing is the lag time between when things happen and when things are printed. In the December issue of this column (“Domino Still Dominates Groupware, but Microsoft Looms Large”), I discussed at great length the price and performance of AS/400s running Domino compared to the competition. When I wrote the column in early October, there were only limited NotesBench test results available on the AS/400, and those numbers did not look very good compared to those of UNIX and NT servers. Since that time, PC server vendors have announced results for servers using Intel Pentium II Xeon chips, and IBM has added results for the AS/400e and RS/6000 line. All the vendors dramatically improved the “bang for the buck” they offer with Domino, but relative market positions based on price, performance, and scalability remain mostly unchanged. While I welcome improved benchmark figures for V4R3 machines, as do people following the midrange computing industry and customers who buy midrange equipment, I think that by charging too much for hardware (relative to other midrange machines), the AS/400 division is missing opportunities to attract new customers. In addition to the improved price and performance of NotesBench, the AS/400 has been shown in a recent study by International Data Corporation (IDC) to offer competitive total cost of ownership (TCO) compared to PC servers running Novell NetWare or Windows NT Server. If publicized by IBM, which is keen on keeping its Netfinity sales increasing as well, the IDC numbers could offset the initial price disparity in hardware and help make a case for the AS/400.

To its credit, the AS/400 division has been focused on scalability for 1998 and
1999. That’s what the Northstar servers are about. Nonetheless, I still have a bone or two to pick with Big Blue. IBM has long been boasting that the AS/400e S40-2208 server could support over 27,000 Domino Mail users. That’s more than three times the number of users for any machine tested. While this is true, it gives the wrong impression that the AS/400 is more scalable than any machine out there. IBM’s S/390 servers could handle as many or more Mail users had they been tested. (They were not tested because they would show truly abysmal price and performance.) Similarly, Hewlett-Packard’s V2250 and Sun Microsystems’ Enterprise 10000 UNIX servers can also handle about the same number of Domino users at about the same price as the S40 configuration. The new 32-way V2500 servers from HP, just announced in December, have at least twice the power of the AS/400e Northstars. Moreover, the RS/6000 S70 Northstar server running AIX was tested soon after the AS/400e S40, and, for half the money, Big Blue’s UNIX techies got 28,800 Domino Mail users on the box, a figure almost identical to that of the S40. It seems IBM’s bragging rights don’t last when it comes to the AS/400.

The fact is that scalability is no longer the issue, because even unimpressive PC servers can support thousands of end users. According to surveys of the AS/400 base in North America, only about a tenth of the companies using 9406 machines have more than 1,000 employees. They can support all of their Domino email users on a model 170 Invader box. Only about half of the 9406 base are companies with between 100 and 500 employees, and another quarter consist of shops with under 100 employees. These companies can get by with a more modest AS/400e server for Domino or a small slice of their production machine.

Even the reliability argument wears thinner and thinner. Fault tolerance is built into Domino to prevent crashes even when running on Windows NT. However, OS/400 has features missing in NT that allow a crashed Domino partition to restart automatically and would make it much more attractive. Still, PC server hardware is so inexpensive that companies can double up on hardware, lash their machines together with Wolfpack or some other clustering technology, and have decent reliability. (I agree this added complexity seems suicidal. However, that doesn’t stop hundreds of thousands of data processing managers from choosing it compared to the tens of thousands who choose an AS/400 each year, even though, technically speaking, an AS/400 Domino server is much better than a PC Domino server.)

What matters in 1999 and 2000 is price, and IBM had better hop to it. The Northstar S40 that handled 27,030 Mail users cost $1.6 million, or $57 per Mail user, which is significantly down from last year’s $137 per user on an Apache S40 12-way server, which supported only a little more than 10,000 users. Recent tests on a Northstar Invader model 170-2292 had a more reasonable cost of $26 per user for 1,350 users. (With the processor, memory and disk price cuts IBM announced in early February to coincide with the Northstar 7XX announcements, the cost of this model 170 configuration dropped to $23 per Mail user.) The RS/6000 S70, which has much less disk and memory and a price $1 million lower than the S40 Northstar machine, can support 28,800 users and costs $21 per user, still a little less than for the AS/400e 170 Invader box. And the older RS/6000s, using 32-bit PowerPC chips rather than the new 64-bit Northstar chips, can handle as many or more users than the Invaders at substantially lower costs. (See Figure 1 for details.) Perhaps more chilling is the fact that IBM’s own Netfinity 7000 M10 server, with its four Pentium II Xeon chips, can handle 6,100 Mail users at $11 per user running Windows NT. A Netfinity 5500 server, with two Xeon chips, can handle 3,400 Mail users for $12 per user. Compaq, Dell, and Hewlett-Packard have similarly powered high-end PC servers, and all three of these vendors plus IBM offer Invader-class Pentium boxes that can support Domino Mail for under $10 per user. Even with the price cuts in February as a result of the Northstar announcements, AS/400s that run Domino will likely be five to six times as expensive as PC servers that will ship alongside them in the early summer.

All Is Not Lost

As always, the AS/400 has an edge in TCO that exists even if it is difficult to quantify. I know that; you know that. However, can IBM communicate that message to a new customer who looks only at the whopping AS/400 price tag and cost of hardware upgrades and figures that either NT or UNIX is a better deal? IBM needs to quantify the lower TCO that AS/400s provide, even if that means establishing an industry benchmark to measure it. For now, AS/400 sales reps and data processing managers who are fighting to keep Domino on the AS/400 or move it to one will have to rely on a TCO analysis by IDC that compares the cost of running Domino on the AS/400 with the cost of running it on a PC server that uses either NetWare or NT.

IDC did in-depth interviews with 30 data processing organizations that use Domino. These organizations used a variety of PC, UNIX, and AS/400 server technology in their production environments and, for one reason or another, chose to support Domino either on the AS/400 or on PC servers. IDC got 15 sites for the AS/400 and 15 for PC servers. Most of the AS/400 shops were running Domino on a model 150 or 170 server; most of the PC sites had Domino on a uniprocessor Pentium II server. IDC then worked some magic on each surveyed company’s budget and user numbers, normalized them for what it would cost to support 1,000 Domino users on their platforms, and averaged them for each platform. IDC found that, over the course of three years, the AS/400 hardware cost $63,000, Domino-related software cost $108,000, and Domino support staff cost $210,000 for a grand total of $381,000. For PC servers, hardware cost a piddling $9,000, software cost $114,000, but support staff cost $342,000 for a total of $465,000 over three years. In addition, the AS/400 had an 18 percent lower TCO than PC servers. The main

reason is that companies supporting Domino on the AS/400 had lower help desk costs, smoother upgrades, and fewer slowdowns in service.

The factor that weighs most heavily in favor of the AS/400 but is not in these TCO figures is the cost of downtime. Even with clustering, which costs little on PC servers, these machines are nowhere near as reliable as standalone AS/400s when it comes to running Domino. That is a very strong marketing message but may not be heard if the AS/400 sales representative can’t get through the door because data processing managers don’t want to pay a lot for their hardware or, more importantly, because they think their nerds are smart enough to beat the reliability and downtime numbers that will most likely end up beating them. Domino users will never know how easy it is to use an AS/400 unless IBM can either turn up the volume on its sales pitch or cut AS/400 prices to get attention.