Two weeks ago, IBM revealed information about how its eServer i5 performs against Windows and Linux servers on an industry benchmark. That benchmark says a lot about the claims that Big Blue can rightfully make about the i5. On closer inspection, it also reveals some claims that the company cannot make...at least, not yet.

The benchmark to which I am referring is the two-tier Sales and Distribution (SD) test from SAP. As its name implies, the SAP SD benchmark tests the ability of a server to process SAP transactions that include the opening of customer orders, the creation of invoices, and changes to existing customer orders. On the benchmark, the i5 demonstrated that it can outperform other servers in its class.

Sizing Up the Competition

For the benchmark, IBM pitted its two-way eServer i5 Model 520-0905 against several two-way servers from Hewlett-Packard, Sun Microsystems, and Dell. Most of these servers were running Windows Server 2003 on Intel's 32-bit Xeon DP processors, though the Sun server was running SuSE's Linux Enterprise 8 on AMD's 64-bit Opteron processors.

As the following table shows, the i5 Model 520 outperformed two-way servers from all three of IBM's top competitors. In the case of Dell's PowerEdge 2650, the performance difference was considerable.

As the table demonstrates, the two-way i5 has a performance edge over Intel's fastest 32-bit chips running at almost twice the clock speed of the i5's POWER5 processors. It also outperforms the two-way, 64-bit Opteron server from Sun. These results are not that surprising for several reasons. First, each POWER5 processor has two CPU cores instead of the single cores of the Intel and AMD processors, giving it twice the number of computing engines. The POWER5 processor also has a huge L3 cache that lets it keep data close to those engines. While both the POWER5 and Xeon DP processors have multithreading capabilities (each processor can manage two threads simultaneously), IBM's chip has the edge with its 64-bit bandwidth. Indeed, given these strengths, it is somewhat surprising that the i5 did not outperform these servers by a wider margin. This may reflect Big Blue's relative lack of experience in tuning POWER5 servers for various workloads. Over time, the company will probably do a better job of optimizing the i5 for public benchmarks.

While the SAP SD benchmark attests to the performance of the i5, it says nothing about the price/performance of the server. However, anyone who understands how to configure these servers can get a general idea of where the i5 stands on that metric. For instance, the minimum configuration of the i5 520-0905 Standard Edition has a list price of $104,000. When you add the memory and disk storage needed to create the configuration that IBM used for the SAP SD benchmark, the list price rises to slightly more than $160,000. This means that the two-way i5 Model 520 probably costs around $370 per SAP SD benchmark user.

By contrast, the cost per SD user is much lower for the Intel and AMD servers. For instance, you can buy the HP BL20p G2--a blade server--as it was configured for its SAP SD tests for just over $20,000. That price includes an enclosure unit for the BL20p that can support seven additional blades. Throw in a Windows Server 2003 Enterprise Edition license for around $4,000 and a two-processor license to SQL Server 2000 for $40,000, and your total cost is around $64,000. Since the HP BL20p G2 supports 408 SD users, its cost per SD user is around $157. The other Intel and AMD servers have similar per SD user costs.

While the i5 Model 520's competitors may cost less per SD user on paper, it is not clear that they would cost less in actual data centers. In the SD benchmark tests, all of the servers achieved their recorded user levels by running at more than 95% CPU utilization rates. Would any of us run a Windows or Linux server at such a utilization rate, particularly on a mission-critical workload such as an SAP system? Of course not. We probably would not run an i5 at such rates either, but most of us would feel comfortable running SAP on an i5 or iSeries at much higher utilization rates than on a Windows or Linux server. That is largely because the i5 and iSeries have the necessary workload and systems management tools to support higher utilization rates at acceptable reliability levels.

As a consequence, the effective cost per SD of the i5 is probably much closer to that of Windows and Intel servers than the theoretical cost per SD. Unfortunately, the industry's benchmark tests fail to make this clear. Given this fact, it would greatly benefit IBM if it were to educate decision-makers about how to compare benchmarks that pit the iSeries and i5 against Windows and Linux servers. Such education could make the i5 more of a contender in competitive bidding environments.

IBM could also benefit from educating customers about the i5's superior ability to run multiple applications at high CPU utilization levels. While Windows and Linux servers may have superior price/performance to the i5 when running single workloads, they still cannot manage multiple workloads with the same agility. As a result, their price/performance curves plunge as workloads are added. Because of this, I would encourage IBM to give its Three-in-One Benchmark--or a similar multi-workload benchmark--to a standards body to administer, and then challenge Windows and Linux server vendors to beat the i5's price/performance on the test. Such a move could help the IT market understand how much of a value the eServer i5 really is.

Lee Kroon is a Senior Industry Analyst for Andrews Consulting Group, a firm that helps mid-sized companies manage business transformation through technology. You can reach him at This email address is being protected from spambots. You need JavaScript enabled to view it..

Lee Kroon is a Senior Industry Analyst for Andrews Consulting Group, a firm that helps mid-sized companies manage business transformation through technology.

Also Read

The Effects of COVID-19 on the IBM i Ecosystem

Dissecting the HelpSystems IBM i Marketplace Survey

IBM i Receives Package Manager Support