Thu, Jun
3 New Articles

Truth in Numbers: The Data Governance Process

Compliance / Privacy
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

When data goes "wrong," it's time to mount a strategy.


Where is the truth in numbers? Who is responsible when data discrepancy occurs? How can upper management be certain that what they see reflected in their reports accurately represents the truth of what is found in any particular manufacturing process, inventory shelf, sales achievement, or future projection?


These questions are particularly important as organizations use business intelligence (BI) applications to pull together the data generated by separate software packages into a global picture of the enterprise as a whole. And until these questions are addressed through data governance, no one can vouch that the numbers they are using represent a picture of the reality that your management can believe.


Data governance is often seen as a "standards" issue: making sure that a widget is called a widget, and a sale is a real sale. But it's much more.


Data governance is a process by which the various management information systems within the organization (engineering, manufacturing, sales, marketing, etc.) can be brought together using a common vernacular with a common understanding of what is required. It's a powerful process that is used to normalize both the information and the expectations that drive the organization toward its strategic goals.


For instance, one department may calculate quantities in stock based upon a part number, while another may count quantities based upon a bill-of-materials formula, while still another uses a number based upon projected estimates of production quotas. Each variation creates a discrepancy that has meaning for one information system but is meaningless to the others. When these three quantities definitions are rolled up into a company's business intelligence suite as metadata, suddenly there are four definitions, making the entire information system appear suspect. Multiply that dynamic across the numerous BI systems that may exist simultaneously in individual departments and/or subsidiaries, and you can start to imagine how the Babylonians felt, building their famed tower.

Supply and Demand Approaches to Data Governance

Why can't IT solve this problem? Well, it's complicated.


IT is often tasked with the development of data definitions as new databases are created or as new information systems come online. But there are problems when this approach is too closely tied to the individual application (both new and legacy) that IT is managing.


When IT defines the data, we end up with—in essence—a supply-side process, because IT is defining both the capabilities and the limitations of what is captured, structured, and reported. This supply-side orientation has many potential problems:


  • IT may have established the initial standards for the format and structure of the information, but these standards may not represent what is actually needed by the management of the organization. Regardless of how closely IT works with line managers in accomplishing the systems analysis, there will be discrepancies between what is needed and what is defined.
  • Many decisions are typically made on information and data outside the systems where the standards have been implemented. That is, the interpretation of the data occurs by individuals who don't know how the data was acquired, compiled, manipulated, or changed.
  • Many systems within our organizations may have duplicate, stale, or inconsistent data. This results in conflicts, confusion, and misinterpretation of data.
  • As data spreads across multiple systems, it is often separated and isolated from the systems that created it and from other iterations that follow. The scores of redundant copies of data and the business processes that use the data can get tangled. Too often, there is little cross-organizational collaboration. Some of this data is structured in databases, such as DB2, but a tremendous amount will be unstructured, housed in spreadsheets, in email systems, or in the cloud.
  • Controls established by IT over the information may ultimately end up being counter-productive to needs of the business as it evolves. This problem is exacerbated as systems mature, as new systems are added, and as personnel and processes change over time.
  • The policies for proactively managing the use of information may never become firmly established, resulting in the use of information in ways that exceed the accuracy or the intent of the information systems that created it.


These problems—often attributed to standards ­problems—are really data governance supply-side problems: IT sets and maintains the standards without oversight on how the information is ultimately used.


Real data governance goes beyond IT's initial involvement and should extend into the management circle of the organization itself. It should extend to the demand side, where individuals and departments collaborate to deliver a clear and accurate picture of the information in the form that management can actually use.


Demand-side data governance is designed to enhance the quality, availability, and integrity of a company's data by fostering cross-organizational collaboration and structured policy-making. It balances departmental silos with organizational interest, directly impacting the four factors any organization cares about most:


  • Increasing revenue
  • Lowering costs
  • Reducing risks
  • Increasing data confidence


Building Demand-Side Governance

Creating a demand-side governance system requires the creation of an ongoing governance team—individuals tasked with guiding the organization to develop the data infrastructure, ensuring that the right questions are being asked as well, and making certain that the information is consistent across the entire infrastructure. It requires moving beyond the processes of program analysis and systems analysis that typify most IT development initiatives. It requires true governance over what information systems should deliver. A good governance team not only identifies the problems and discrepancies, but also has the authority to request IT to make changes to the systems involved. It is charged with these tasks:


  • Investigating the problems that obstruct management from receiving correct, timely, and critical information
  • Analyzing how the current and proposed systems interface with management's requirements
  • Proposing solutions and additions to the current information infrastructure
  • Achieving the cost/benefit analysis to obtain management go-ahead
  • Budgeting the costs associated with changes or additions to the systems
  • Evaluating the impact of implemented projects
  • Communicating the value to the management and the organization as a whole


Of course, different organizations treat governance differently. Many larger organizations have a dedicated staff composed of experts in a both IT and the organization's business model, with comprehensive responsibilities to management to ensure that goals are achieved.


Other organizations have teams composed of assigned members from each department or a subsidiary who meet regularly and chart the critical flow of information.


Still others try to support data governance through design teams, led by IT itself.


However, and in order to maintain momentum with management, it's first vitally important to understand where your company's current information processes fit in the scheme of data governance.

What Is the State of Your Data Governance?

There are guidelines to help measure how well your organization is approaching the discipline of data governance.


In 1984, the Software Engineering Institute (SEI) created the Capability Maturity Model (CMM) as a methodology used to develop and refine an organization's software development process. The CMM describes a five-level graduated path that provides a framework for prioritizing actions, a starting point, a common language, and a method to measure progress. Ultimately, this structured collection of elements offers a steady, measurable progression to the final desired state of fully mature processes.


The Data Governance Council adapted the CMM into a Data Governance Maturity Model (DGMM). Using this model, an organization can identify where it stands with data governance and chart its trajectory based on its own business needs.


An organization assesses the level of governance maturity within its organizational structures by identifying the following milestones and benchmarks:


  • Level 1: Policies around regulatory and legal controls are put into place. Data considered "critical" to those policies is identified. Risk assessments may also be done around the protection of critical data.
  • Level 2: More data-related regulatory controls are documented and published to the whole organization. There is a more proactive approach to problem resolution with a team-based approach and repeatable processes. Metadata becomes an important part of documenting critical data elements.
  • Level 3: Data-related policies become more unambiguous and clear and reflect the organization's data principles. Data integration opportunities are better recognized and leveraged. Risk assessment for data integrity, quality, and a single version of the truth becomes part of the organization's project methodology.
  • Level 4: The organization further defines the "value" of data for more and more data elements and sets value-based policies around those decisions. Data governance structures are enterprise-wide. Data governance methodology is introduced during the planning stages of new projects. Enterprise data models are documented and published.
  • Level 5: Data governance is second nature. ROI for data-related projects is consistently tracked. Innovations are encouraged. Business value of data management is recognized, and cost of data management is easier to manage. Costs are reduced as processes become more automated and streamlined.

Moving up the Model

Odd as it may seem, some smaller organizations are closer to achieving Level 5 of the Data Governance Maturity Model, because they have only one database or fewer applications to normalize. And as we all have experienced, management has a proclivity to believe that adding that next suite of applications (MRP, ERP, SC, BI) will finally give it the ultimate view of the organization's data. In fact, what IT knows is that new applications are, instead, merely one more version of information, and often this version is at odds with previous versions.


That's where data governance begins to earn its value. It studies the realm of possibilities from management's perspective and then focuses on achieving the normalized view of the organization that management demands.


Data governance is a process, not an ultimate goal, and the team that you develop can use the DGMM as a guideline to help communicate to management where your organization fits and what the industry feels is the pathway to better governance.

Assembling the Data Governance Team

OK, so you see your company's governance problems. Now what?


While IT cannot and should not be the arbiter of a data governance process, it can definitely take the lead in establishing the framework and assembling the team. But IT needs management support to achieve critical mass within the organization. It needs to transform the process from a simple technical standards issue into an organizational collaborative opportunity between systems and silos. Having a "management champion" is a key to the success of this endeavor: someone whose success is also tied to the truth-in-numbers that data governance promises.


When you find that first candidate champion within your company, don't start by talking about the issues or the cost/benefits; instead, ask what's required to help that person work more effectively. You'd be surprised how many simple/obvious things have never been addressed. Be the concerned consultant, and then ask your management champion for suggestions about how you might obtain some expert personnel resource from his team to achieve the necessary depth of understanding of the problem.


Start on a manageable problem, and map how that problem connects to problems within other management areas. Use this same champion strategy to begin assembling a team of experts until you're comfortable that you have the right people, ones who will be dedicated and who can be assigned to achieve the results that their bosses require.


Then set the first agenda items and arrange the initial meeting. Lay out the turf, explain the connected problems, and lead them to the first solution. But remember: you are the spark plug in this process, not the engine.


More than likely, your experts will need further resources and management commitment to fully understand the dimensions of the larger problems they are addressing, but try not to let any one team member become bottle-necked with responsibilities. Data governance is a collaborative learning process for everyone, so monitoring and measuring how the process is progressing is important.

Collaborate and Communicate

When you've nailed the parameters of a solution, have the team—not merely the IT department—report back to your management champions. Encourage them to give frank appraisals, demonstrating the savings that can be achieved and/or the efficiencies that can be activated. Let your champions determine the next steps to achieving problem-solving goals within their own team of management co-workers. And then wait for their permission to proceed.


Over time, as each data governance problem is addressed, don't hesitate to promote the success of the team structure to other members of management. Soon you'll undoubtedly discover others within management that have similar issues. Those issues should be brought back to the data governance team.


Let the team expand organically with the experts who can help you identify the requirements. Make certain that expert team members whose immediate problems have been resolved don't retire without an appropriately experienced replacement.

Achieving Truth in Numbers

Bit by bit, the benefits of a data governance team will begin to penetrate the individual silos of information of your organization. Bit by bit, the truth-in-numbers arguments and confusions that afflict the information system will begin to disperse. And in the process, your organization will increase its effectiveness, the knowledge of its experts, and the respect of your company as a whole.

as/400, os/400, iseries, system i, i5/os, ibm i, power systems, 6.1, 7.1, V7,

Thomas Stockwell

Thomas M. Stockwell is an independent IT analyst and writer. He is the former Editor in Chief of MC Press Online and Midrange Computing magazine and has over 20 years of experience as a programmer, systems engineer, IT director, industry analyst, author, speaker, consultant, and editor.  


Tom works from his home in the Napa Valley in California. He can be reached at ITincendiary.com.





Support MC Press Online

$0.00 Raised:

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: