This article series is designed to provide you with an introduction to the basic concepts of artificial intelligence (AI), machine learning (ML), and deep learning (DL), as well as introduce you to some of the terminology that is frequently used in discussions regarding this technology. This is not a deep dive into AI, ML, and DL but rather a “dictionary” of AI, ML, and DL topics and terms.

The information provided is NOT intended to make you an AI, ML, or DL expert. Nor will it show you how to build, train, and deploy ML and DL models. Instead, it will give you a good foundation in AI, ML, and DL and help you understand some of the terminology you are likely to encounter in discussions, presentations, papers, and books on the subject.

Understanding data

Figure 1: "Data is the new oil"

In 2006, Clive Humby, a British mathematician and entrepreneur in the field of data science and customer-centric business strategies, coined the phrase “Data is the new oil”. Soon after, Michael Palmer, executive vice president of the Association of National Advertisers, amplified the point in a blog post, writing that like oil, data is “valuable, but if unrefined it cannot really be used. [Oil] has to be changed into gas, plastic, chemicals, etc. to create a valuable entity that drives profitable activity; so, data must be broken down and analysed for it to have value.”

But how do organizations deal with the massive amounts of data that’s available to them? And how can they change this valuable “crude” into an even more valuable commodity – the insight needed to make actionable decisions?

These questions are even more consequential today than they were in 2006, given the exponential increase in the amount of data being generated. According to Statista (https://www.statista.com), approximately 97 zettabytes of data was created, copied, captured, or consumed around the world in 2022. And this number is expected to grow to 181 zettabytes by 2025! Therefore, it is incumbent upon organizations to employ tools like artificial intelligence (AI), machine learning (ML), and deep learning (DL), which enable them to ingest and process all this information, and in turn, gain insights that will help them make better decisions and ultimately improve their bottom line.

In its most basic form, data is information that has been translated into some format that can be easily stored, moved, or processed. Computers represent data as binary values using patterns of just two numbers: 0 and 1. And, keeping with the categorization that is central to most computer operations, data can fall into one of three categories:

• Structured
• Unstructured
• Semi-structured

Technically, all data has structure of some sort. So, deciding which category certain data belongs to often comes down to whether that data has a pre-defined structure or is organized in a pre-defined way.

Sources:

Data, the “New Oil,” Cannot Be Used If Left Unrefined. Thankfully, AI Can Help
https://donbasile.com/2021/05/28/data-the-new-oil-cannot-be-used-if-left-unrefined-thankfully-ai-can-help/

Why Do People Still Think Data Is the New Oil?
https://datainnovation.org/2018/01/why-do-people-still-think-data-is-the-new-oil/

Structured Data

Figure 2: Structured data – data that has a clearly defined format

Structured data refers to data that has a clearly defined size and data type – commonly used data types include characters, numbers, dates, times, and groups of characters called strings. Usually, structured data is organized into some type of formatted repository (typically a spreadsheet or database), so its elements can be made addressable for more effective processing and analysis. Most experts agree that structured data accounts for about 20% of all the data that is generated.

If you look at an Excel spreadsheet, you will notice that it consists of rows (identified by numbers along the left-hand side) and columns (identified by characters across the top). A column typically represents some attribute that has a well-defined data type, and a row represents a single record that has multiple columnar values (or attributes). For instance, a record (row) that contains bank account information, might have one column that contains the account number, another column that contains the account balance, another column that contains the date the last debit or credit was made against the account, and so on. The column that contains the account number may be defined such that it only accepts a string of characters and numbers, the account balance column may only accept currency values, and the column containing information about dates that transactions were performed may only accept valid dates.

As you can see from this example, each row in a structured data set is well defined in that all column values for any given row are somewhat related. We say the row is “structured” because there is a well-defined column for every value stored in the row and each row contains the same set of columns.

Relational database management systems (RDBMSs) like IBM Db2, Oracle, Microsoft SQL Server, and so forth, store data using a similar row-column format, but at a much larger scale. Consequently, they often serve as the main repository for structured data in a company and are frequently used as the primary data source for AI, ML, and DL projects. In many cases, such projects are looking for patterns in existing rows so they can predict the value for a given column.

For example, if a table in a database contains information about the local real estate market, a single record (for a house) might contain information such as style, location, square footage, lot size, number of bedrooms, number of bathrooms, room sizes, tax appraisal value, last sale price, and so on. Using this data, an ML model could be developed that accurately predicts the sales price for a home when presented with information found in the other columns. And an application could be developed that uses this ML model to predict the optimum sale price for a new home coming on the market when all relevant information is provided.

 Unstructured Data

Figure 3: Unstructured data – drawings, photographic images, audio clips, videos, etc.

Unstructured data is data that either does not have a pre-defined data model or is not organized in a pre-defined manner. Drawings, photographic images, audio clips, and videos are examples of unstructured data. That said, it’s important to note that a small amount of metadata – attributes that define the data itself – that has a defined structure may be stored with this type of data so delivery mechanisms can use it. For example, image data stored in a JPEG (Joint Photographic Experts Group)-formatted file will have some structured metadata stored with it so select applications will know how to use the data to render the image on a computer screen.

According to Gartner, somewhere around 80% of enterprise data originates in unstructured form. IDC claims that unstructured data grows at 26.8% annually compared to structured data, which only grows at 19.6%. And it’s not just the digital photos, recordings, videos, and social media postings people produce daily that is driving this trend; unstructured data is increasingly being generated by machines like laboratory equipment and medical devices, as well as software that aids in tasks like electronic design and geospatial modeling.

As businesses embrace AI, ML, and DL, unstructured data is becoming essential to their offerings and operations. Why? Because ML and DL algorithms are capable of gaining valuable insights from unstructured data that are otherwise impossible to derive.

For example, the medical industry is beginning to see ML and DL models that can use unstructured data in the form of images to help diagnose cancer and other diseases. In other areas, ML and DL models are being used to search social media for patterns that indicate a user’s sentiment (how they feel) about a particular product or service, and to create chatbots (software that conducts a conversation via audio or text to convincingly simulate how a human would behave as a conversational partner) to answer questions that traditionally would have been addressed by someone working at a help desk.

Semi-structured Data

Figure 4: Semi-structured data – data with some number of organizational properties.

Semi-structured data is a form of structured data that does not adhere to the formal structure used with spreadsheets and relational databases, but nonetheless has some organizational properties like semantic tags and hierarchies, that make it easier to read and analyze. A good example of semi-structured data is Hypertext Markup Language (HTML) code, which doesn't restrict the amount of information that can be stored in a document, but still enforces a hierarchy via required semantic elements. (HTML is the standard markup language that is used for documents that are designed to be displayed in a web browser.) Other forms of semi-structured data include:

• Email messages – a method of exchanging messages between people using electronic devices.
• Comma Separated Value (CSV) files – CSV files consist of “rows” of data where each “column” value is separated by a comma and each row is separated by a carriage-return/line feed character.
• Extensible Markup Language (XML) documents – XML is a markup language that defines a set of rules for encoding documents in a format that is both human and machine readable.
• JavaScript Object Notation (JSON) objects – JSON is an open standard format that uses human readable text to transmit data objects consisting of attribute–value pairs.
• Electronic Data Interchange (EDI) documents – EDI is a process that allows one company to send information to another company electronically rather than with paper. Many business documents can be exchanged using EDI, but the two most common are purchase orders and invoices.
• Resource Description Framework (RDF) metadata – RDF is a standard model for data interchange on the Web that allows structured and semi-structured data to be mixed, exposed, and shared across different applications.
• NoSQL database data – A NoSQL database provides a mechanism for the storage and retrieval of data that is modeled in methods other than that used with relational databases. (Popular NoSQL databases include MongoDB, Redis, Apache CouchDB, and Cassandra, to name a few.)

It’s important to note that because very little data in the modern age has absolutely no structure and no metadata associated with it, data that is categorized as being unstructured could, technically, be considered semi-structured. So, there are some who argue that the distinction between unstructured and semi-structured data is moot.

Adding Dimensions to Data

Figure 5: Data in terms of dimensions

Now that we have looked at the different types of data that’s available, let’s talk about data in terms of dimensions. It’s relatively easy to conceptualize dimensions when it comes to two-dimensional (2D) and three-dimensional (3D) objects. For example, a circle, square, or triangle drawn on a piece of paper has two dimensions – width and height. But a cube or sphere drawn on a piece of paper, along with the paper itself and practically every other object in the world has three dimensions – width, height, and depth (sometimes referred to as length or thickness).

It’s also not that difficult for humans to visualize objects in four dimensions. The four-dimensional example shown in Figure 5 begins with a three-dimensional graph that shows X, Y, and Z planes, and then adds color to create a fourth dimension. In the real world, the fourth dimension is often referred to as the time dimension. Thus, a box being thrown from one person to another has four dimensions – the width, height, and length of the box, coupled with the time it takes for the box to travel through the air.

But how do you graphically represent a fifth-dimension, sixth-dimension, or more? If you have data stored in a table that has four or more columns and you think of each column as being a different dimension, it’s not too difficult to picture what data with more than four dimensions might look like. However, trying to find patterns or correlations in all that data is difficult. And having more than four columns (dimensions) makes the problem exponentially more difficult to solve. The more dimensions you have, the more combinations of data you need to examine to find patterns amongst all the possible combinations of data available. That’s why it’s a good idea to eliminate dimensions that are not needed before you attempt to use data for AI, ML, and DL.

Stay Tuned

In the next part of this article series, we’ll explore what AI, ML, and DL are, as well as look at the building blocks of AI, ML, and DL, and explore the typical ML and DL workflow.