During the development cycle of an IT system, there are many times that data needs to be integrated. Examples include:
- CI Data (computers, servers, switches (and all related information)
- Managers (org chart-ability)
And a myriad of other information. When IT is consuming this data via a connection of some sort I basically see the sources falling into a few categories listed in order of rarity.
- Sources that just work (plug-and-play)
- Sources that are total crap (can’t integrate, export, or flex the data in any way shape or form)
- Sources that work
And that last one is a tricky son of a gun, and like I said, it’s likely the most common. In that data type the 90-10 rule applies two ways:
- 90% of the information is accurate and good
- 90% of the information will easily import
And what these two things mean are that your system integration people are likely spending 90+ percent of their time chasing down that last 10% of information. Whether it’s finding a better source, cleaning up data, or working with stubborn data, this can be a huge time waste.
And why is it?
The systems we are working with are usually owned by IT (spoiler, this is why), and we are essentially our own customer in many regards, likely some other aspect is dependent on another group, i.e. HR process making sure employee data is correct in Active directory.
Here is what I propose; rules for good future data integration planning:
- Always list your first requirement on an RFP as “ease of integration”
- Never select a system, tool or service that doesn’t integrate well (read reviews, talk to your architects, talk to current customers)
- Make sure your requirements list out the specific integration types that need to be built, and be specific i.e. - the system shall provide a real-time ODBC connection
- Include specific integration details in your design documentation i.e. the SQL query “X” should return “Y”
- Test, test and re-test
- All data points need robust process to keep data up to date and relevant (yes I’m glossing this, but that’s a different post methinks)
- This is the important one: hire architects who care, can say no, have the power to say no, and are involved in each and every project from day one.
All of this basically comes back to one point, which is that all your data systems will need to talk together someday. Maybe not today, and maybe not tomorrow, but someday these things will need to talk to each other. Making it easy to integrate, consume and query this data from many different angles, using many different methods, will cut major costs on projects and major time/effort burn on your development staff.