A set of tenets can help deliver successful data and AI software projects. Each tenet is a continual journey, not a destination. Each must be practiced until it becomes second nature, and each tenet has multi-dependencies with other tenets.

There’s a set of symbiotic tenets and disciplines when it comes to building and delivering software products that can help drive success for both vendors and clients alike. Here are what I consider to be some of the top tenets.

Innovation

One of the ways to differentiate a product or service in the marketplace is to introduce new ideas, methods, processes, technologies. This also includes “disruptive technologies” that can redefine the market by disrupting the status quo and establishing a new norm. Organizations that invest heavily in research and patents have the potential to lead or redefine the market on their terms. But innovation is more than just technology. It is also heavily influenced by the culture to cultivate innovative thinking and an entrepreneurial spirit. It also requires leaders to innovate. Innovation leadership can support achievement of the mission and the vision of an organization or group. With new technologies and processes, it is necessary for organizations to think innovatively to ensure continued success and stay competitive to adapt to new changes. Innovation, like all the other tenets, is a journey.

Incubators and Startups

Incubators (first appearing in 1959) were created to help entrepreneurs solve some of the problems commonly associated with running a startup by providing workspace, seeding funding, mentoring, and training so that entrepreneurs could focus on growing their business without the worry of the usual business overheads. Large or well-established companies often try to promote innovation by setting up "internal startups," allowing new business divisions to operate at arm's length from the rest of the company. However, these can fail for numerous reasons, including “corporate interference.”

While the vast majority of startups fail, some become “unicorns,” privately held startup companies valued at over US $1 billion. Internal startups may also lack incentives or rewards (financial or otherwise) to succeed, whereas external startups often have high risk with the chance of very high return. That said, there are many lessons to be learned by large corporations from the successes of external incubator startups.

Simplicity

Simplicity is somewhat of a dichotomy in that it can often be elusive and very difficult to attain. Innovation and sophistication do not necessarily imply complexity. Simplicity in itself can be the elegance of a product. Take, for example, the creation of the iPod, a high-tech yet intuitive, simple-to-use device that enabled consumers to carry their entire music collection with them. It was a breakthrough disruptive technology that stole the market from other media vendors focused primarily on sound fidelity.

As products evolve over their life cycle, complexity can increase, making it challenging for those involved in testing, management, and support, which in turn can negatively impact the customer experience. Convergence can facilitate simplicity through reuse (discussed later). Take, for example, the strategy of developing a set of common services that can be consumed across a portfolio of offerings. These (and all other services) need to conform to and inherit a series of specifications forming a “bedrock” for future product offerings to be based on. Convergence rather than divergence and uncontrolled variation should be the mantra of any organization involved in software product development.

Quality

What makes us prefer one particular cola drink, a specific vehicle manufacturer, or a certain clothing brand? While there might be one major factor, there are usually multiple factors or dimensions that help us make our choices, all of which add up to an overall satisfaction or quality score. Over time, our satisfaction with a product or service may increase or decrease due to good or bad experiences. If delighted, we might review the product/service and leave 5 stars. If dissatisfied, we might leave a scathing review and 1 star. The quality of a product or service can not only directly impact its reputation, it can also impact the vendor’s reputation, brand value, and ultimately the bottom line. While there are costs associated with good quality, poor quality often has even greater costs. From a product-development perspective, quality is a team sport (multi-dimensional) in that some aspects of a product or service may be weighted differently depending on the environment within which it is used. Quality is everybody’s responsibility and should be embedded within every aspect of a product’s lifecycle.

Reliability and Resiliency

Reliability is a key consideration when purchasing a product or service. When part or all of a critical computer system or application fails, the result can be loss of revenue, lost business opportunities, or risk of exposure, such as a security breach. It can also damage brand reputation. Headlines of data security breaches involving Personally Identifiable Information (PII) have caused many consumers to no longer trust the institution that had a data loss. Reliability is also closely associated with resilience, the ability or capacity to recover quickly from difficulties. This is also a key attribute for how well a business-critical system can detect and avoid failure or its ability to recover from a system failure. Elimination of single points of failure, failure detection and prevention, and reliable crossover are part of high-availability computing. For example, customers use the mainframe because they trust it to protect their data and run some of their most business-critical applications. Imagine if every application, service, process, and data store could leverage those same characteristics.

Agility—Not Speed, Agility

Compared to traditional software engineering, agile software development mainly targets complex systems and product development with dynamic, non-deterministic and non-linear characteristics. Accurate estimates, stable plans, and predictions are often hard to obtain in early stages of a project, so agile practitioners seek to reduce the leap of faith that is needed before any evidence of value can be obtained. Requirements and design tend to be emergent as large deterministic specifications might not be economically viable. Lessons from years of successes and failures have helped shape agile development's favor of adaptive, iterative, and evolutionary development. In this day and age, with dynamic turbulent markets, organizations need to have business agility to adapt and respond to emerging opportunities with competitive differential. Adopting agile development practices can help.

Reuse, Don't Rebuild

In my experience, products can become more complex as they evolve over time, particularly when we don’t share ideas, or common functions, methods, and processes. We often resort to the “Not Invented Here” syndrome. Sometimes we perceive existing capabilities as not good enough and reinvent the wheel. This can cause divergence, which can lead to unnecessary cost and complexity. Reuse can result in savings. For example, a microservices architecture embraces reuse. Monolithic code can be decomposed into a series of smaller services––building blocks (like Lego bricks) to recompose new trusted services reusing much of what already exists––maintained and controlled through a services catalog. Open source also promotes reuse, and for that reason some organizations adopt the open-source model within their own enterprise––“inner source” for want of a better phrase. Moving from siloed development to inner source and microservices can help organizations coalesce innovation and investment streams. The hope would be that organizations can potentially increase the quality of their software and achieve greater agility for their software developers.

Open

The word “open” in the software world is much more than just open source. It also refers to software that has open, accessible, or published APIs. It can also mean open as in transparency and explainability—a key focus area in artificial intelligence that warrants its own blog post. But let me focus on open source for the purpose of this article.

Things are rapidly evolving, and it's become clear to me that no single company can create AI solutions in a vacuum and hope to keep pace with the rate of change in such a fast-growing segment of the tech industry. Open source–based infrastructure services such as storage access services, managing of security services, and managing containerized microservices can help create greater portability of offerings by isolating the specific dependencies of the underlying computing environment while avoiding the perception of “vendor lock-in.” Finally, by appropriately engaging with the open-source ecosystem, vendors can gain insight into the areas that developers are most interested in and better understand where the marketplace is going.

Cloud: Hybrid by Design

Cloud computing relies on the sharing of resources (services) to achieve coherence and economies of scale with the potential for improved manageability and less maintenance. Cloud in general is seen as attractive by organizations of all sizes because it can help minimize up-front IT infrastructure costs with different pay-as-you-go and subscription models. Hybrid cloud is a computing environment that connects an organization’s on-premises private cloud services and third-party public cloud services into a single, flexible infrastructure for running the organization’s applications and workloads. It should give an organization the flexibility to choose the optimal cloud for each application, data source, or workload (and to move workloads freely between the two clouds as circumstances change). This enables the organization to meet its technical and business objectives more effectively and cost-efficiently than it could by forcing an organization to choose between a public or a private cloud. This necessitates that the entire software stack by design needs to be able to run transparently in hybrid cloud environments, with common DevOps, governance, security, and management capabilities that span multitenant, dedicated/isolated configurations. Hybrid and multi-cloud support has become an essential design tenet. The future is truly hybrid.

“X” as a Service

Everything in cloud computing should be defined as a service, whether it is infrastructure, platform, software, data, compute, security, or others. These are collectively denoted as XaaS. While some organizations choose to manage their applications and data on-premise or on their own private cloud, other organizations may want to pay for services as and when needed. For example, Software as a Service (SaaS) is a software licensing and delivery model in which software is licensed on a subscription basis and is centrally hosted. It is sometimes referred to as on-demand software. It behooves software vendors to offer their portfolio of software––from individual services and applications to (for example) entire data and AI platform––as on-demand services. This approach enables vendors to engage organizations of any size and maturity to leverage their offerings within an organization’s cost and operating constraints. As organizations grow, they have the benefit of choosing to host and manage those same services on their own private cloud. Vendors that can offer that flexibility stand to serve a wider range of customers, potentially expanding the user base.

Summary

In summary, while each tenet can be valuable on its own, the whole is greater than the sum of its parts. I have concluded that there is a common theme across all of the above tenets. Each is a continual journey, not a destination. Each must be practiced until it becomes second nature, and each tenet has multi-dependencies with other tenets.

Procrastination is not an option. The journey must begin. Those that hesitate may risk losing to their competition.

Let me leave you with a quote from Artificial IntelIigence: Evolution and Revolution, a book I was privileged to coauthor: “A journey of a thousand miles begins with a single step.”

Mark Simmonds is a Program Director in IBM Data and AI communications. He writes extensively on machine learning and data science, holding a number of author recognition awards. He previously worked as an IT architect, leading complex infrastructure design projects. He is a member of the British Computer Society and holds a Bachelor’s Degree in Computer Science.

---

MC Press books written by Mark Simmonds available now on the MC Press Bookstore.

		Artificial Intelligence: Evolution and Revolution Get started on your AI journey with insights for a path to success. List Price $19.95 Now On Sale

Also Read

Upskilling and reskilling for talent transformation in the era of AI

Dispelling Some Fog About Edge Computing

Laying the Corporate Cultural Groundwork for an AI Project