The AS/400 RAISE software integration test is the final test that IBM performs on a new release of OS/400 prior to general availability (GA). As a member of the AS/400 RAISE test team, I can tell you that our approach is similar to most system integration tests, but our focus goes beyond just verifying that the products work together. Our mission is to use and drive the AS/400 system like real customers do by creating an environment with multiple AS/400 systems in a network performing CPU-intensive applications as well as interactive jobs. We run our tests as if we were a business in production running 24 hours a day, seven days a week.

RAISE stands for reliability, availability, installability, serviceability, and environment. All of these are important components of our testing, but the heart of our testing is our test environment. Our environment is designed to represent an interrelated set of companies that use the AS/400 to support and drive their day-to-day business activities. Within that environment, test scenarios are defined to simulate different types of user activities, resulting in the interaction of multiple scenarios executing across a network of systems.

The Test Environment

Figure 1 shows a typical RAISE test environment. In this environment, we have an Internet service provider (ISP), a home office, a branch office, traveling employees, and dial-up customers.

To simulate a real-world ISP, we have created a RAISE service provider (RSP). Our RSP contains an AS/400, PCs, and an Integrated Netfinity Server for AS/400 (INS), which uses Windows NT server code. The RSP is its own business entity. The home office and its affiliates have to contract Internet use via the RSP, and access to RSP company resources is not allowed from anywhere in the rest of the network.

The home office serves as company headquarters. It contains seven AS/400 systems of different model types, each with its own purpose. It also has an intranet for employee-to-employee information sharing as well as various types of servers. There are six “logical” systems that can only be reached either via the branch office or by traveling employees. Three of these systems use Logical Partioning (LPAR) and share a 12-way processor. The Home6 system contains a firewall to keep it secure when coming through

Router A, and Home7 can be accessed by the general public, the branch office, traveling employees, or dial-up customers.

The branch office is a subsidiary of the home office. The two offices are geographically separated, but the branch office can access the home office systems through either an ISDN leased line to the Home5 system or the ISDN switched line that goes through Router A. In the latter case, a virtual private network (VPN) is used to connect the branch office to the home office. The VPN provides a secure connection.

Traveling employees are employees who work from the home, branch, or RSP office. Traveling employees can either connect to the RSP via an analog switched line or access the home office systems by using the analog switched private line to Home5.

Dial-up customers are Internet users and have access to the RSP via an analog switched line. From the RSP, dial-up customers can reach the unsecured Home7 system via the ISDN switched line. (The firewall prevents access to the Home6 system.)

Test Scenarios

Instead of using typical test scenarios that focus on verifying specific functions, we use test scenarios that consist of several fictitious companies that use the AS/400 to run their businesses: a parent company, a telemarketing company, an airline reservation service, an investment firm, and a World Wide Web company.

The parent company owns and manages the other companies and uses the AS/400 database to store employee, department, and payroll information. It is our oldest set of test cases, and, for the most part, the test cases are run interactively with some batch jobs. The parent company uses RPG, COBOL, C, and CL applications to run the payroll and update employee and department information.

The telemarketing company provides order entry, sales, and telemarketing support. It uses the AS/400 database to store catalog, price, inventory, and customer information in addition to customer orders. Telemarketing applications contain a mixture of interactive and batch test cases and use both Original Program Model (OPM) and ILE development environments. The company originally used RPG, COBOL, C, and CL for its business applications but recently updated some applications to use Java.

The airline reservation service uses the AS/400 database to store airport, airline, and flight information and customer orders and data. It uses Web technology such as Common Gateway Interface (CGI) and Net.Data to allow customers to book flights via Web pages and maintains agent rates for booking flights. The agent rates are then tied into the payroll application of the parent company.

The investment firm simulates a banking and investment enterprise. It is a startup company that uses servlets, Enterprise JavaBeans (EJBs), JavaServer Pages (JSP), and Java Database Connectivity (JDBC) to provide Web pages that allow customers to check account balances, view stock history and business reports, buy and sell stocks and mutual funds, and perform other banking functions.

The World Wide Web company simulates the sale of consumer products over the Internet. It uses an AS/400 Web server with Net.Commerce to sell its products and an AS/400 firewall to protect the company’s private network and data from Internet hackers. Customers can load the company’s home page, shop around, make some purchases, or send email concerning their orders to store employees. The AS/400 Web server processes customer requests, and the AS/400 database stores product information, customer orders, and inventory.

The Test Cycle

Each of these fictitious companies provides services to fictitious customers. As a testing team, we serve as IT specialists who create and maintain business applications, as employees who run the day-to-day businesses, and as customers. In other words, there are several “roles” that we take on and walk through during a test cycle.

We start as IT specialists who look at new AS/400 product offerings and determine which of our businesses could or should take advantage of the enhancements. One of the most difficult tasks we have is determining what we should test. It is impossible for us to utilize the AS/400 in all the ways our customers do. Therefore, we focus on common business practices utilized as well as on strategic solutions. We use all the resources available to us to determine what we should test. This includes working with our system architects and designers and talking with customer groups such as the Large User Group (LUG).

Once requirements are defined and applications are created, we take on the roles of employees and customers. These roles are defined for each of the businesses. For example, the World Wide Web company has customers who purchase merchandise over the Internet and employees located in the home and branch offices. As customers, we can shop but do not have access to company networks or private company systems. As home office employees, we process orders as well as manage the AS/400 servers and support the company’s private network and systems. Finally, as branch office employees, we use SOCKS servers, proxies, and VPNs to access private company systems.

Our test cycle starts with just getting our scenarios to run successfully. As things progress and system stability improves, we increase the number of employees and customers on the systems. At this point, the workload team starts running test scenarios in the background to increase the load on the system. A portion of the workload test cases provide a general system load by performing typical system activities such as Display Job (DSPJOB) or Work with Spool Files (WRKSPLF). In addition, the workloads provide automation and simulation. Using automation tools such as WinRunner, we can take a specific scenario, capture the keystrokes, and play them back. Automation allows us to walk through scenarios without human intervention but allows us only one virtual user per PC. To simulate multiple users, the workload team uses tools such as LoadRunner, which allows us to simulate multiple users running the applications and create high network traffic and server loads.

The goal is to have no unplanned IPLs. Using the workloads, we put the systems into full production for one month. This allows us to simulate an active business and detect problems that might occur only after the systems and network have been operating for an extended, uninterrupted period of time.

Our Work Is Never Done

Like any business, the AS/400 RAISE team continues to make changes and updates; we are never totally satisfied with our environment. As testers, we want to try new things and see what we can break. As the AS/400 continues to grow and expand, so will the AS/400 RAISE test environment. We will continue to look for ways to improve our test environment and ensure that we provide the best possible product.

HOME OFFICE

Internet Service Provider Dial-up Customers

BRANCH OFFICE
Traveling Employees

Home7 AS/400

Ethernet Router A Firewall

Token-Ring Branch

AS/400

ISDN Switched to Router A

ISDN Switched Analog Switched

Home6 AS/400

Ethernet or Token-Ring Home5 AS/400

ISDN Leased to Home5

ISDN Switched ISDN Switched to Branch

Public Branch Exchange (PBX)

Analog Switchedto Home5 or ISP Analog

Switched to ISP

ISDN Leased to Branch

Analog Switched

Figure 1: The AS/400 RAISE test environment simulates a customer environment.