Automating build of Integration Services projects

After our experiences with the build process in Analysis Services – expectations were low when we turned our gaze towards Integrations Services (SSIS).

Our overall experience with SSIS has generally been positive – it’s an extremely powerful tool with some interesting performance characteristics. There are some annoying flaws – especially in the Visual Studio integration. When I work with SSIS projects – three or four daily crashes of Visual Studio is not uncommon. Never fear – the power provided by the tool more than makes up for this when it comes to choosing the ETL platform. And buggyness is what service packs are for … isn’t it? 🙂

The first thing you notice when you choose "build" in Visual Studio on an SSIS projects is that Visual Studio doesn’t do anything – except of course copy files to the bin directory. Why? Well, guess you could say that the *.dtsx files contain all the information need to run the packages – there is "nothing to be done".

Automating the building of BI-Solutions

Coming from a "Old fashioned" development environment (that is: C/C++, CSV, Unix and the likes) I have always wondered why BI-projects don’t seem to get the idea of a structured software development.

There are some technological cornerstones of a good, well structured software development process:

• Source control systems – Never loose code again, unlimited undo. Tag your releases
• Automated, daily builds – Make sure you project is ready to deploy and release. Never again spend hours on manually building your project
• Bug tracking and issue logs – Speaks for itself. A necessary tool for proper project management
• Multiple environments – At the very least: Development, Test and Production.
• Change management – Well documented and scripted procedures for moving your code into production

I don’t know about you – but I have yet to see a Business Intelligence projects use all of the above. Some, actually a lot, of the examples I have worked on did not even use ANY of the above. My theory is that BI-development community is in a sense "immature" to the ways of structured software development.