Skip to content

Category Archives: etl

New recruit to my ETL toolbox

I’ve recently completed my first real DataStage project and took a chance to get certified while all the stuff is still “fresh”. Certification itself is quite complex and I didn’t use most of the tricks depicted in questions up until the moment when one of the jobs had to process a quarter billion of rows [...]

Posted 17 December 2011 † ykud § ‡ Comments (2) °

Using session variables in Cognos BI

Just a quick Cognos BI hint: you can use session variables to store project-level constant values. I’m a big fan of ‘feature-rich’ ETL reports showing not only what dimension element mismatch between systems, but also allowing seamless editing of element mapping. This usually means drilling down from report into external application for dimension mapping. Parameters [...]

Posted 22 August 2011 † ykud § bi § ibm ‡ Comments (0) °

ETL Testing

I’m a fan of testing in each DWH project, because it allows to: – be sure that at least tested parts work - change logic w\o retesting all results Receiving “All OK” in the morning means that there’s time for a cup of coffee ) A list of approaches I use for testing ETL-procedures: ‘Water in [...]

Posted 24 September 2010 † ykud § ‡ Comments (2) °

Freebase GridWorks — a data-driven approach to ETL

Take 5 minutes to watch screencast for FreeBase Gridworks — an interesting new approach to data transformation. Instead of ‘transformation-based’ approach of every tool on the market, this tool uses ‘data-based’ approach, which looks rather intriguing. Especially ‘Undo-Redo’ function ) I keep thinking when to use this tool. User driven data load in DWH? This [...]

Posted 11 May 2010 † ykud § ‡ Comments (0) °