Wed, Jun
4 New Articles

Instrumentation and Automation Fuel Customer Experience Data Collection

Commerce - Other
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

Editor's Note: This article is an excerpt from the book Customer Experience Analytics published by MC Press.


Large automation investments in customer touch points, products, and business processes are rapidly creating enormous volumes of data as well as capabilities to make changes at electronic speeds. This chapter explores these changes in "instrumentation" and how they provide an environment for Customer Experience Analytics (CEA).


IVR, kiosks, mobile devices, email, chat, corporate web sites, third-party applications, and social networks have generated a fair amount of event information about our customers. In addition, customer interaction via traditional media, such as call centers, can now be analyzed and organized. The biggest change is in our ability to modify the customer experience using software (policies, procedures, and personalization), making self-service increasingly customer-friendly. The availability of raw data about customers provides an unprecedented opportunity not only to analyze and understand the customer but also to adaptively change customer-facing systems and processes to improve customer experience.

Sales and Marketing

Analytics related to customers and their experiences has been a widely studied area for over 50 years. Most of the core marketing science and related disciplines developed models of customer behavior, devised ways to measure customer experience, and used analytics to peek into the decision-making process. While academics worked on very impressive customer models and techniques for analysis and prediction, it was often difficult to apply these principles in practice, because of a lack of and high cost of data collection.


The good news is that all the work in marketing science can now be applied as the data becomes available. To that definition, I would like to add the tasks associated with data ingestion, categorization, and management to support the analytics.


Sales and marketing got their biggest boost in instrumentation from Internet-driven automation over the past 10 years. Browsing, shopping, ordering, and customer service on the web has not only provided tremendous control to end users; it has also created an enormous flood of information to the marketing, product, and sales organization in understanding buyer behavior. Each sequence of web clicks can be collected, collated, and analyzed for customer delight, puzzlement, dysphoria, or outright defection and the sequence leading to this decision.


Self-service has crept in through a variety of means: IVRs, kiosks, handheld devices, and many others. Each of these electronic means of communication acts like a gigantic pool of time-and-motion studies. We have data available on how many steps a customer took, how many products she compared, and what she focused on: price, features, brand comparisons, recommendations, defects, and so on. Suppliers have gained enormous amounts of data from self-service, electronic leashes connected to products, and the use of IT. If I use a two-way set-top box to watch television, the supplier has instant access to my channel-surfing behavior. Did I change the channel when the advertisement started? Did I turn the volume up or down when the jingle started to play? If I use the Internet to shop for a product, my click stream can be analyzed and used to study shopping behavior. How many products did I look at? What did I view in each product? Was it the product description or the price? This enriched set of data allows us to analyze customer experience in the minutest detail.


What are the sources of data from such self-service interactions?


  • Product—As products become increasingly electronic, they provide a lot of valuable data to the supplier regarding product use and product quality. In many cases, suppliers can also collect information about the context in which a product was used. Products can also supply information related to frequency of use, interruptions, usage skipping, and other related aspects.
  • Electronic touch points—A fair amount of data can be collected from the touch points used for product shopping, purchase, use, or payment. IVR tree traversals can be logged, web click streams can be collected, and so on.
  • Components—Sometimes, components may provide additional information. This information could include data about component failures, use, or lack thereof. For example, a wireless telecommunications provider can collect data from networks, cell towers, third parties, and handheld devices to understand how all the components together provided a good or bad service to the customer.


As much as we have used instrumentation to collect rich amounts of customer data, CEA can also be used to drive a new set of behaviors. Over the past 30 years, we have seen gradual maturing of our understanding of CEA and how it impacts sales and marketing. The early evolution was in use of CEA for segmentation. The original segmentations were demographic in nature and used hard consumer data—such as geography, age, gender, and ethnic characteristics—to establish market segmentations. Marketers soon realized that behavioral traits were important parameters to segment the customers.


As our understanding grew, we saw more emphasis on micro segments—specific niche markets based on CEA-driven parameters. For example, marketers started to differentiate innovators and early adapters, as compared with late adapters, in their willingness to purchase new electronic gadgets. Customer experience data let us characterize innovators who were eager to share experiences and were more tolerant of product defects.


In the mid-1990s, with automation in customer touch points and use of the Internet for customer self-service, marketing started to get interested in personalization and 1:1 marketing. As Martha Rogers and Don Peppers point out in their book The One to One Future, "The basis for 1:1 marketing is share of customer, not just market share. Instead of selling as many products as possible over the next sales period to whomever will buy them, the goal of the 1:1 marketer is to sell one customer at a time as many products as possible over the lifetime of that customer's patronage. Mass marketers develop a product and try to find customers for that product. But 1:1 marketers develop a customer and try to find products for that customer."


Early CEA systems were reporting systems that provided raw segmentation data to the marketing team so that they could use the data to decide on marketing activities, such as campaigns. Automation in marketing and operations gave us the opportunity to close the loop—use CEA to collect effectiveness data to revise and improve campaigns. We are seeing surges in campaign activity. Marketers are interested in micro-campaigns that are designed specifically for a micro-segment or, in some cases, for specific customers. The customer experience information gives us criteria for including a customer in the campaign.


For example, prepaid wireless providers are engaging in micro-campaigns targeted at customers who are about to run out of their prepaid minutes. These customers are the most likely to churn to a competitor and could easily continue with their current wireless provider if they were to be directed to a store that sells prepaid wireless cards.


Another area of interest is Next Best Action (NBA)—in other words, recommending an activity based on the customer's latest experience with the product. This could include an up-sell/cross-sell based on current product ownership, usage level, and behavioral profile. NBA could be offered any time the sales organization has the opportunity to connect with the customer via a touch point. NBA is far more effective in sales conversion compared with canned rules that repeatedly offer the same product over and over across a customer interaction channel. (Imagine your airline offering you a discounted trip to your favorite warm-weather golf vacation spot on a cold day.) NBA can also be revised based on feedback from customer reaction.


Pricing has been a hotly pursued topic for business, as each percent increase in price without a corresponding decrease in demand means an increase in profits. There has been a growing trend to use price optimization models—mathematical programs that calculate how demand varies at different price levels—and then combine that data with information about costs and inventory levels to recommend prices that will improve profits. Given the complexity of pricing and the thousands of items in highly dynamic market conditions, modeling results and insights helps to forecast demand, develop pricing and promotional strategies, control inventory levels, and improve customer satisfaction.



Arvind Sathi

Dr. Arvind Sathi is the Global Communication Sector Lead Architect for IBM's Information Agenda team. He received his PhD in Business Administration from Carnegie Mellon University and worked under Nobel Prize winner Dr. Herbert A. Simon. Dr. Sathi is a seasoned professional with more than 20 years of leadership in Information Management architecture and delivery. His primary focus has been in the delivery and architecture oversight of IT projects to communications organizations. He has extensive experience with many domestic and international communications service providers and other services industries.

Before joining IBM, Dr. Sathi was the pioneer in developing knowledge-based solutions for CRM at Carnegie Group. At BearingPoint, he led the development of Enterprise Integration, MDM, and Operations Support Systems/Business Support Systems (OSS/BSS) solutions for the communications market and also developed horizontal solutions for communications, financial services, and public services. At IBM, Dr. Sathi has led several Information Management programs in MDM, data security, business intelligence, and related areas and has provided strategic architecture oversight to IBM’s strategic accounts. He has also delivered a number of workshops and presentations at industry conferences on technical subjects including MDM and data architecture, and he holds patents in data masking.

MC Press books written by Arvind Sathi available now on the MC Press Bookstore.

Big Data Analytics Big Data Analytics
Get a practitioner’s advice on how to succeed with Big Data analytics.
List Price $16.95

Now On Sale

Customer Experience Analytics Customer Experience Analytics
Improve customer relationships, products, and processes with CEA.
List Price $19.95

Now On Sale



Support MC Press Online

$0.00 Raised:

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: