Technology has its limits

As this month's article reminds us, data integration is hard work. As much as we'd like it to be true, the latest and greatest tools are not a short-cut.

For more on data, see my April DM Review article It Is all About the Data and my most recent column, The Data Shadow System Conundrum.

I hope you enjoy the articles and feel free to let me know what you think.

And if you have suggestions for future topics for this newsletter or the Data Doghouse blog, feel free to share them.

Can Technology Cure All That Ails Us?

Rick Sherman, Athena IT Solutions

Each week millions of people watch the TV series CSI: Crime Scene Investigation or one of its offspring, CSI: NY and CSI: Miami. The premise of each show is that a group of crime scene investigators use their high-tech gadgets to solve even the most mysterious of murder cases. Using cool tools they can uncover obscure evidence that reveals the true story.

Technology plays a starring role in these shows -- it always saves the day. And that's a theme in many American TV shows, movies and books, which is appealing to those of us who work in the IT industry.

The problem with these shows is that they set a false expectation about technology. If cool tools can solve murder mysteries on TV, why can't they solve murder mysteries in real life? Can't the police just call in their CSI unit to figure things out?

It's a similar phenomenon in the IT marketplace, where companies are constantly inundated with new products, product upgrades and acronyms. They're told that the latest and greatest technology is going to save the day by solving data problems that have plagued them for decades. Corporate performance management (CPM), master data management (MDM) and customer data integration (CDI) are the latest to join the fray, promising an immediate single version of the truth simply through the use of the right tools. Mystery solved!

Vendor pitches can be very convincing on PowerPoint slides and in marketing brochures -- especially when customer references and proofs-of-concept reinforce their points. BI, ERP, CRM, CPM, extract, transform and load (ETL), enterprise information integration (EII) and enterprise application integration (EAI) vendors cite examples of their products solving a company's data problems in a matter of hours. It's no wonder companies are lulled into thinking that data integration success is relative to the tool that is selected.

Be realistic.

In reality the single version of the truth takes hard work — very hard work. If it was only a technology issue many companies would have solved it by now. After all, they've spent a lot of money implementing the latest and greatest technology and what do they have? More data silos than ever before.

Yes, technology can help, but the issues are much deeper and more profound that that. For true data integration it is essential to know what the data represents, when it was captured, how it was transformed and how it was manipulated in the report or analysis performed by the business user. Companies need to ensure data consistency, integrity and quality, and they should be able to track the data from its creation to its consumption.

Tools are only part of the strategy. The other part involves talking to business people to understand and define the data and its usage; creating an architecture to ensure data integration with integrity, consistency and quality; effectively managing resources; training users; and educating both the business groups and the IT organization on what it all means. These steps will help when it comes to making the business case to justify the necessity of data integration.

It's difficult work, but consider some of the outcomes and advantages:

• Easing Sarbanes-Oxley compliance.
• Providing financial transparency.
• Improving interactions with suppliers and partners.
• Improving customer intelligence and customer relationships.

Remember, technology is not a solution — it's an enabler. Don't ignore the data integration fundamentals. Attractive dashboards and "slicing and dicing" cubes are terrific, but without the right data, your system is doomed. Sure, it's impressive to load millions of rows from dozens of source systems in real time. But if you've scrimped on the architecture and the data's consistency, integrity and quality are questionable, all you've got is a fast garbage truck.