Typically, I find that my processing jobs break down into 4 parts:<\/p>\n
- \n
- setup (typically job entries)<\/li>\n
- loading data to a stream (extract)<\/li>\n
- processing that data (transform)<\/li>\n
- saving that data to a persistent datastore (load)<\/li>\n<\/ul>\n
Often, I combine the last two steps into a single transformation.<\/p>\n
So, for this sample project (final code is here<\/a>), we will create a couple of transformations containing business logic.\u00a0 (All transformations are built using Spoon on Windows with Pentaho Data Integration version 4.4.0.)<\/p>\n
The business needs to greet people appropriately, so our job will take a list of names and output that same list with a greeting customized for each person. \u00a0This is the logic we are going to be testing.<\/p>\n
First, the skeleton of the code that takes our input data and adds a greeting.\u00a0 This transformation is called ‘Greet The World’.<\/p>\n
![\"pentaho-basic-logic-75\"](\"http:\/\/www.mooreds.com\/wordpress\/wp-content\/uploads\/2013\/04\/pentaho-basic-logic-75-300x168.png\")
<\/a><\/p>\nI also created a ‘Load People to Greet’ transformation that is just a text file input<\/a> step and a copy rows to results<\/a> step.
![\"pentaho-basic-logic-load-75\"](\"http:\/\/www.mooreds.com\/wordpress\/wp-content\/uploads\/2013\/04\/pentaho-basic-logic-load-75-300x168.png\")
<\/a><\/p>\nThe last piece you can see in this is the ‘GreetFolks’ job which merely strings together these two transformations.\u00a0 This would be the real job that would be run regularly to serve the business’ needs.<\/p>\n
![\"pentaho-basic-logic-job-75\"](\"http:\/\/www.mooreds.com\/wordpress\/wp-content\/uploads\/2013\/04\/pentaho-basic-logic-job-75-300x168.png\")
<\/a><\/p>\nThis logic is not complicated, but could grow to be quite complex.\u00a0 Depending on the data we are being passed in, we could grow the logic in the ‘Greet The World’ transformation to be quite complex–the variety of greetings could depend on the time of year, any special holidays happening, the gender or age or occupation of the person, etc, etc.<\/p>\n
Astute observers may note that I didn’t write a test first.\u00a0 The reason for this is that getting the test harness right before you write these skeletons is hard.\u00a0 It’s easier to write the simplest skeleton, add a test to it, and then for all<\/strong> future development, right a failing test first.<\/p>\n
As a reminder, I’ll be publishing another installment of this tutorial in a couple of days.\u00a0 But if you can’t wait, the full code is on github<\/a>.<\/p>\n