In 2008, companies will be working harder to develop a culture of innovation in order to drive the market. Having access to internal and external information will be critical to this effort.

We worked through 2007, and, with the exception of a long and grisly war, it was a pretty good year. Looking ahead to 2008, we see signs that there may be an economic slowdown spawned in large measure by the sub-prime mortgage meltdown. Regardless of whether that comes to pass, the U.S. has assumed a huge national debt that will have to be paid off essentially by the nation's business output. You wonder why they keep letting immigrants into the country? Hey, we need as many taxpayers as we can get! Can people work any harder? Maybe, maybe not. But they sure could work a lot smarter.

James Todhunter, chief technology officer at Invention Machine, a leading innovation software company, argues that companies will have to do a better job of connecting the dots when it comes to implementing the ideas of their employees. Workers must come up with a regular stream of great ideas, and those ideas must make their way into the companies' products and services if the company expects to survive and thrive in a down market. He puts forth a number of principles necessary for companies to develop a culture of innovation, including one key element—that innovation starts at the top and management must "make its commitment to innovation clear and unambiguous." Others include these:

• Provide innovation best-practices training.
• Reward innovation contributors.
• Invest in the right technology infrastructure to support innovation best practices.
• Give innovation workers visibility into corporate objectives so their ideas support the company's direction.
• Continuously communicate the value of innovation successes to clients and the company.
• Practice innovation in everything workers do.
• Make available highly precise concept retrieval tools that are integrated with innovation practices.

It is this last item that may be of most interest to IT administrators since what Todhunter is talking about here is today's generation of advanced search tools that can pull out information from unstructured as well as structured data. With the change in rules of civil procedure making companies responsible for producing relevant email when subpoenaed by a litigant, the need to be able to access the mountains of information stored in corporate computers takes on a whole new meaning. The regulation would be just another costly burdensome compliance requirement if it weren't for the fact that the information that is locked away on these computers could produce bottom-line benefits to the company if workers could only access to it. Needless to say, there are privacy issues, and I don't want my colleague snooping around notes I sent to my girlfriend. But in large corporations, there are gigabytes of data locked away in emails and other unstructured data repositories that could, if mined, produce benefits to both customers and the company. Knowing what is there, how valuable it is, and how to access it falls within the realm of IT.

The field of search technology has advanced significantly in the past year. If 2007 was a good year for anything, it was this, and MC Press Online has carried a number of stories about the product breakthroughs in this field during the past year. Todhunter points out that an important part of the innovation infrastructure is the "framework to leverage knowledge." By "knowledge," he means information both within the organization as well as outside the enterprise. And, he cautions, "don't make the mistake of assuming that a traditional knowledge management initiative will meet the specialized needs of your innovation workers." Designers, engineers, and scientists need better tools.

One initiative that is bearing fruit in giving knowledge workers better access to the vast repositories of information available to them is the open-source framework IBM created to help companies build new analysis technologies. Called Unstructured Information Management Architecture, or UIMA, the framework is designed to foster tools that will allow companies to realize more value from their unstructured information by discovering relationships, identifying patterns, and predicting outcomes. UIMA is now an open-source project at the Apache Software Foundation and already is used extensively to enable text analysis, extraction, and concept search capabilities among IBM's OmniFind enterprise search portfolio. The portfolio includes products with names like OmniFind Enterprise Edition, OmniFind Analytics Edition, and OmniFind Yahoo! Edition.

The latest tool to be announced by IBM is called IBM OmniFind Personal Email Search, or IOPES. For Lotus Notes and Outlook, it is a powerful "smart" search tool that allows people to retrieve information from their emails in a way that goes far beyond results produced by keywords. The software is powered by advanced algorithms that can interpret incomplete queries and find information such as phone numbers, people, meetings, presentations, documents, images, and more.

Available for free from IBM alpha Works, the software was the brainchild of Shiv Vaiyanathan, manager of unstructured information mining at IBM Research. Vaiyanathan, as the story goes, was searching for a phone number that a colleague had sent to him in an email. Fruitless attempts to plug in the sender's name and the word "phone" produced nothing but irrelevant, and somewhat frustrating, results. Vaiyanathan and a couple of colleagues began working on a better way, and they programmed in common search concepts such as dates, times, and phone numbers. A collaborative effort spanning IBM research labs in Almaden, California ; Haifa, Israel ; and Delhi, India , helped refine IOPES. The software was tested inside IBM by early adopters. The company has a program called Technology Adoption Program whereby employees try the products that come out of R&D and provide developers with their feedback. It's now available to users at large, and the hope is that, by offering it for free on alpha Works, IBM will continue to receive feedback. At some point, IBM may release it as a commercial product for searching intranets, at which point it may start charging for it.

Douglas Wilson, chief technology officer of Lotus, is apparently quite enthusiastic about IOPES, saying, "OmniFind Personal Email Search illustrates how IBM's advanced technology delivers the ability to quickly and easily access the precise information we need, exactly when we need it." Wilson notes that with gigabytes of email storage available to everyone, "email has evolved from a simple communication tool into a personal database where we retain vast amounts of valuable information."

Making the information locked away in a company's computers available to those who can use it effectively and learning how to leverage that information to benefit not only the company wherein it is contained, but also customers it is serving will be a significant challenge over the coming year. It will be up to IT administrators to evaluate and implement the tools that knowledge workers need to enhance their company's innovation results so that 2008 can be as successful as—or even more successful than—2007.