Mon, Jul
4 New Articles

Technology Focus: IBM i BI in the Era of Big Data

  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

Big Data is big buzz these days, but has this new IT fad really made it to the IBM i? Reports from business intelligence (BI) software vendors are trending toward "not so much."


To many, the idea that "if accurate data is good, large volumes of accurate data is better" is a self-confirming truism. At the very least, that sentiment is a driving factor behind Big Data's popularity as a concept. You need look no further than IBM's own Big Data page to see how its boosters view the idea.


That Big Data is at least on the horizon for the i seems certain. On February 25, IBM announced that SoftLayer Technologies will incorporate IBM Power Systems into its cloud infrastructure, based on IBM Watson services, which will boost SoftLayer's Big Data services and extend them to the IBM i. So we know IBM is busily preparing for a day when Big Data will be in demand among i users.


But is it here yet? Like many IT trends over the years, there are some indications that this one seemingly hasn't made as deep an impression on enterprises supporting the i as it has on users of other platforms. Opinions from several major BI software vendors for the i platform seem to confirm this idea.

What Is Big Data?

"Big data is generally defined as a high volume of dynamic, structured, and unstructured data being extracted and stored in real-time or near real-time from multiple sources," offers Bill Langston, director of marketing at New Generation Software. "It's intended to help an organization spot trends, opportunities, risks, or other insights they couldn't see if they were only looking at their internal production data. Much of the complexity in big data comes not just from the volume the name implies, but the variety of data formats being captured and analyzed."


However, Langston goes on to say: "I think 'big data' projects are rare in the IBM i market because so many IBM i customers are B2B wholesale distribution and manufacturing companies, supply chain partners, and small companies serving well-defined customer sets. You typically wouldn't find much, if any, useful unstructured data in the chatter on Facebook, LinkedIn, or Twitter about the products, services, or business practices of these kinds of companies."


Marcel Sarrasin, director of marketing for Quadrant and BCD Software, generally agrees with Langston's assessment of Big Data in today's i market. "Our customers don't really use the words 'Big Data' even though many are creating IBM i web reports over files with millions of records. Maybe Big Data is too much of a buzzword for them. Our customers mainly create reports like dashboards with drill-downs to see the details. The data is usually from DB2 with some people using MS SQL or MySQL; however, we aren't seeing as much need for unstructured data, which is often a part of Big Data. It also doesn't seem like they are getting data at the same velocity as what is often depicted as Big Data, from things like sensors or web server logs."


"The data you care most about in a B2B company is in your operational CRM and ERP systems," elaborates NGS' Langston. "However, there is more and better public data becoming available all the time, and there is definitely opportunity for creative thinkers in B2B companies to augment their internally generated data with potentially useful data sources like the US Census, local government building permits and real estate transactions, automotive registration records, marriage licenses, weather history, commodity prices, currency exchange rates, crime reports, and a myriad of others. Expanding your company's database assets to external sources like these reflects the spirit of Big Data thinking, even if doing this doesn't take a company outside the familiar world of relational data you can easily store in DB2 on i. This really isn't a new concept, but it's more practical today due to lower storage costs and better availability of higher-quality data."


A third BI software vendor, which requested anonymity when contacted for this article, simply stated flatly, "We do not have Big Data customers on the i."

Big Data on the IBM i

That doesn't mean Big Data projects are totally alien to the i platform. Mike Hennel, President and CEO of Silvon Software, reports there are Big Data projects active among Silvon's customers.


"The Big Data projects that we're working on right now with our customers revolve around three key areas: Point-of-Sale (POS) data, metrics related to the performance of their B2B and B2C e-commerce websites in driving visitor purchases, and the building of global repositories of internal and external data from which visibility into business performance can be assessed on more of a worldwide (versus regional or country-wide) scale," he notes.


"One of our more interesting Big Data projects has been for a fast-moving consumer-goods client who is now capturing POS data from retailers around the world to better understand global sales trends," Hennel elaborates. "They're one of our first customers to employ social metrics to assess consumer sentiment at the brand level. Plus, the company is now analyzing its internal marketing programs and other external events on a worldwide basis to assess the impact of such events on product sales. Taking such a global view of the business involves the dynamic capturing and analysis of extremely high volumes of data coming from a very large number of internal and external sources."

How Big Is Big?

From these comments, and if the market as a whole is in any way typical of what these vendors have shared with us, it's possible to think that "Big Data" could be somewhat dependent on the eye of the beholder. One could argue that if an enterprise thinks its project involves Big Data, perhaps it does even if the project's data universe doesn't necessarily follow a "textbook" definition of Big Data. Such ambiguities have often been typical of technology adoption practices in the past for the i, which would be to say there are a few early adopters who are embracing Big Data, but the bulk of the i community remains on the outside looking inso far.


Whether that's truly the case now is impossible to determine from the few data points we've reviewed here. But with IBM's interest in the Softlayer Technologies agreement and some evidence of i users carrying out Big Data projects already, it's probably safe to say the IBM i BI market is, at best, still entering a transitional phase when it comes to Big Data.


Nevertheless, as Big Data's appeal grows and more IBM i users start contemplating taking on projects of that scope, that will slowly generate some pressure for change on BI software producers.


"In the future, our customers will require more specialized BI applications to help them better interpret and understand what's happening with all of the new data that's coming from outside the organization," speculates Silvon's Hennel. "And as they uncover additional insights from this data, they will need software solutions to further leverage the dataas in the area of planning optimization."

BI on the i Today

Whatever the future of Big Data on the IBM i, there are numerous BI tools available today for companies looking to expand their analysis of business data crucial to their enterprises. Below are major products currently available, divided into dashboard and intermediate BI products first, followed by full-blown BI systems compatible with the i. Please note that this list doesn't include BI support products like simple query tools, database maintenance utilities, report generators, and the like. These products have their place but don't offer the range of analysis opportunities that dashboard and full-featured BI offerings do.


Also please note that the product descriptions simply highlight a few features of each product. Be sure to check the links included with the descriptions and contact the individual vendors for a complete picture of what each product offers.

Dashboard and Intermediate Products for IBM i


Attunity Change Data Capture (CDC)

Attunity CDC captures and highlights all changes made to any corporate data sources, including databases, data warehouses, and non-relational data sources. Data sources can reside on IBM i and z systems, HP Nonstop, OpenVMS, UNIX, and Windows platforms.


Attunity Connect

Connect provides universal SQL connectivity via an array of database adapters that connect to a variety of PC, midrange, and mainframe databases and let users make this data available to any Java, .NET, and Web applications. Connect also offers XML data services for relational sources such as DB2/400, DB2, and Oracle, as well as non-relational sources such as text files.


Attunity Federate

Federate lets users build single views of corporate data from the same range of sources as its CDC and Connect products. The product lets users run distributed queries across multiple data sources and access tables from varying data sources as if they were part of a single catalog.


Automation Centre

Tracker Data Warehouse

Tracker Data Warehouse is a browser-based data miner that uses SQL Server as a data store and extracts information for display in spreadsheets, one of 40 preconfigured report templates, or user-designed custom report formats.


BCD Software

Clover Query

Clover Query is a business intelligence tool that enables unlimited users to create real-time web-based reports, charts and dashboards. It includes DB2, MySQL and MS SQL templates so anyone can produce professional looking reports in minutes.


Monarch for EZ-Pickin's

Monarch for EZ-Pickin's is a PC report-miner that lets users translate IBM i and other-platform reports into interactive views or extract data for export to other applications, without re-keying data or further programming.




SEQUEL is a query and reporting tool that delivers data to the user's choice of graphical interfaces, such as dashboards, tables, custom-formatted reports, Web displays, and other media. It also provides options for data cleansing and reformatting, data drill-down, and distribution via email and other means.


IBM Corporation

IBM Cognos Real-time Monitoring

IBM's Cognos Real-time Monitoring displays key performance indicator (KPI) and other business operational data on self-service dashboards to augment other BI data for frontline business users.


IBM SPSS Collaboration and Deployment Services

SPSS Collaboration and Deployment Services is a consultation service that helps enterprises share data assets across multiple departments, enterprises, or other entities. Services include automating analytical processes and deployment of analytic results in existing or custom-built applications and other venues.



LANSA Client

LANSA Client helps non-technical users query, report, and analyze data from IBM i and Windows servers. Users can download the data to MS Excel spreadsheets, graphical views such as charts and graphs, and comma-delimited ASCII text files.



Logi Info

Logi Info lets non-technical users build custom reports, dashboards, Web portals, interactive mobile apps, and other analytical tools using nearly any data source. The product includes more than 600 pre-built elements that can be selected to construct data display and analytical tools without programmer help.


Momentum Utilities


Momentum's inTuator runs on PC desktops but offers multidimensional analysis and charting tools for existing IBM i data on connected servers. Users can link data across application areas, work with downloaded data offline, and export or download data to MS Office applications.


NewGeneration Software (NGS)

Business Performance Dashboard

Business Performance Dashboard lets business professionals view, present, and interact with charts, reports, tables, gauges, calendars, maps, and other visualization aids via a Web browser. Information from these sources is also downloadable to MS Excel spreadsheets without additional programming.


Decision Assist – Business Performance

Decision Assist – Business Performance provides pre-built reports, multidimensional models, dashboards, and a scalable data-warehouse model to extract, transform, cleanse, and present data from IBM i-based distribution applications.



SmartView is a Windows-based, online analytical processing (OLAP) application that uses IBM i data gathered by the NGS-IQ application (see below). SmartView lets users analyze and display data in a wide range of ways, as well as customize it to show specific information types via drag-and-drop formatting aids.


QL2 Software


WebQL is a proprietary language that runs on servers using UNIX or Windows and can harvest nearly any kind of data from Web and non-Web sources (including SQL), handle data transformation and restructuring processes, and then download the collected data to existing internal and external systems.


Visual Mining

NetCharts Performance Dashboards

NetCharts lets executives and other nontechnical users select operational business data and generate dashboards that display and analyze KPIs, reports, and other business metrics.

Full-Service Business Intelligence Products for IBM i

Business Computer Design International (BCD)

Nexus Portal

Nexus Portal provides a prefabricated, customizable Web portal for accessing enterprise information on IBM i machines. Nexus Portal can display information on Internet or intranet portals, provides security features, provides dynamic and menu navigation, and can generate dashboards that focus on KPIs.


Coglin Mill

RODIN Data Asset Management

RODIN Data Asset Management is a data-management environment for data warehouses, datamarts, and other data-integration applications. It includes tools for describing and managing metadata, acquiring new data, handling change-management and versioning, and managing ETL processes. The product is available in four editions tailored to different-sized organizations and includes an optional DB2 Web Query version.



DMA Managed Analytic Applications

Deloitte offers BI analytic applications that can draw data from application systems such as Lawson and SAP, as well as homegrown IBM i-based applications. From this data, DMA Managed Analytic Applications can help users build dashboards, scorecards, user-specific reports, and other aids for analyzing business data.


Dimensional Insight

The Diver Solution

An end-to-end BI solution, Diver includes analysis, integration, and dashboard features, as well as security, real-time alerts, and extensive deployment options. It lets users build and deploy customized BI options using data from IBM i and other data sources.


IBM Corporation

Cognos Business Intelligence

Cognos BI provides reports, analysis, dashboards, and scorecarding to help users capture business data and other metrics to provide a well-rounded view of enterprise activity. It helps users analyze information as well as distribute it via impromptu or standardized reports, dashboards, and other graphical means, including social networking.


IBM DB2 Web Query for i

IBM DB2 Web Query for i, despite a name that makes it sound like a simple query tool, is a BI application that offers support for dimensional analysis, can analyze data from all DB2 and SQL Server platforms, produces reports in user choice of browser or hard copy, and lets users import Query/400 definitions, among other features.


Information Builders

iWay Integrity Solutions

The iWay Integrity Solutions suite provides several products that underlie a strong business intelligence system. These include data quality management (for optimizing data correctness), master data management (for consolidating up to millions of records), and data governance (management and control of data by its internal owners). iWay solutions facilitate real-time management of data from any source.


Performance Management Framework

Performance Management Framework can draw information from more than 300 IBM i and other platform data sources to deliver enterprise performance management information to all levels of business executives. The product includes more than 500 pre-packaged metrics for common business functions and can produce information in a wide array of graphical formats.


WebFOCUS Business Intelligence

WebFOCUS Business Intelligence provides business data for analysis via dashboards, scorecards, query and analysis tools, mobile devices, guided ad hoc reporting, integration with desktop products, and dynamic report distribution with real-time alerts.



WebFOCUS RStat is a predictive analytics and data-mining application for corporate data. It offers a single platform for BI, data modeling, and data scoring that helps users build and export models for prediction and classification, and provides data exploration, descriptive statistics, and interactive graphs.


WebFOCUS Visual Discovery

Visual Discovery is a data-analysis solution with built-in visualization features. It lets users build multidimensional, graphical representations of data and output completed dashboards to MS PowerPoint, Adobe PDF files, and compound WebFOCUS reports.


NewGeneration Software (NGS)

Decision Assist

Decision Assist is made up of prefabricated business analytics and reporting models that specialize in either financial performance or business performance. Designed specifically for i5/OS, it includes a built-in datamart, a Web portal user interface, dashboards and multidimensional models, and predefined spreadsheets and reports.



NGS-IQ is a suite of business intelligence products that includes Decision Assist, IQ Client (a Windows query and Web report developer), Qport Access (data transfer to DB2 databases and MS Office apps), Qport Builder (a Web-based query developer), Business Performance Dashboard (browser-based graphical displays for business data), and IQ Server (a 5250 interface for building queries and interfacing with mobile devices).


NGS-IQ Cloud

NGS-IQ Cloud provides cloud-based BI services built on the company's NGS-IQ product suite. Features include security, query, analytics, reporting, and data replication.



Business Intelligence Solutions for IBM

SAP's products include a portfolio of BI applications that work with DB2 on all IBM platforms, as well as Lotus Notes/Domino, WebSphere portals, Tivoli products, and Informix databases.


Silvon Software

Silvon Stratum

Tailored for manufacturing operations, Stratum offers the ability to analyze data gathered in enterprise resource planning (ERP), customer relationship management (CRM), and Web-based transaction applications. It lets users compare and rank KPIs from different business areas, access hundreds of pre-built views and reports, carry out predictive and budget activities, and deliver information throughout the enterprise.


John Ghrist

John Ghrist has been a journalist, programmer, and systems manager in the computer industry since 1982. He has covered the market for IBM i servers and their predecessor platforms for more than a quarter century and has attended more than 25 COMMON conferences. A former editor-in-chief with Defense Computing and a senior editor with SystemiNEWS, John has written and edited hundreds of articles and blogs for more than a dozen print and electronic publications. You can reach him at This email address is being protected from spambots. You need JavaScript enabled to view it..



Support MC Press Online


Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: