ETL Testing

ETL stands for Extract-Transform-Load and it is a process of how data is loaded from the source system to the data warehouse.  Data is extracted from an OLTP database, transformed to match the data warehouse schema and loaded into the data warehouse database.  Many data warehouses also incorporate data from non-OLTP systems such as text files, legacy systems and spreadsheets. 

Course Duration :

30-35 hours including real time project experience

Pre Requisites

Basic Knowledge about Business Intelligence and Database.

Course Content

Ø  Data ware Housing Concepts

·         What is Data Warehouse?

·         Need of Data Warehouse.

·         Introduction to OLTP, ETL and OLAP Systems.

·         Difference between OLTP and OLAP

·         Data Warehouse Architecture

·         Data Marts

·         ODS (Operational Data Store)

·         Dimensional Modelling

·         Difference Between relation and dimensional modelling

·         Star Schema and Snowflake Schema

·         What is fact table

·         What is Dimension table

·         Normalization and De-Normalization

Ø  ETL Testing

·         ETL Architecture

·         What is ETL and importance of ETL testing

·         How DWH ETL Testing is different from the Application Testing

·         SDLC/STLC in the ETL Projects.

·         Challenges in DWH ETL Testing compare to other testing

o    Incompatible and duplicate data

o    Loss of data during ETL Process

o    Testers have no privileges to execute ETL jobs by their own

o    Volume and complexity of data is very huge

o    Fault in business process and procedures

o    Trouble acquiring and building test data

·         ETL Testing Work flow activates involved

o    Analyse and interpret business requirements/workflows to Create estimations

o    Approve requirements and prepare the Test plan for the system testing

o    Prepare the test cases with the help of design documents provided by the developer team

o    Execute System testing and integration testing

o    Best practices to create quality documentations (Test Plans, Test Scripts and Test closure summaries)

o    How to detect the bugs in the ETL testing

o    How to report the bugs in the ETL testing

o    How to co-ordinate with developer team for resolving the defects

·         Types of ETL Testing

o    Data completeness

o    Data transformation

o    Data quality

o    Performance and scalability

o    Integration testing

o    User-acceptance testing

·         SQL Queries for ETL Testing

·         Incremental load testing

·         Initial Load/Full load testing

·         Different ETL tools available in the market

§  Informatic

§  Ab Initio

§  IBM Data stage

·         Power centre Components

§  Designer

§  Repository Manager

§  Workflow Manager

§  Workflow Monitor

§  Power Centre Admin Console

·         Informatic Concepts and Overview

§  Informatic Architecture

·         Sources

§  Working with relational Sources

§  Working with flat Files

·         Targets

§  Working with Relational Targets

§  Working with Flat File Targets

·         Transformations -Active and Passive Transformations

§  Expression

§  Lookup -Different types of lookup Caches

§  Sequence Generator

§  Filter

§  Joiner

§  Sorter

§  Rank

§  Router

§  Aggregator

§  Source Qualifier

§  Update Strategy

§  Normalizer

§  Union

§  Stored Procedure

§  Slowly Changing Dimension

§  SCD Type1

§  SCD Type2 – Date, Flag and Version

§  SCD Type3

·         Workflow Manger

§  Creating Reusable tasks

§  Workflows, Worklets & Sessions

§  Tasks

§  Session

§  Decision task

§  Control Task

§  Event wait task

§  Timer task

§  Monitoring workflows and debugging errors

·         Indirect Loading

·         Constraint based load ordering

·         Target Load plan

·         Worklet, Mapplet, Resuable transformation

·         Migration – XML migration and Folder Copy

·         Scheduling Workflow

·         Parameter and Variables

·         XML Source, Target and Transformations

·         Performance Tuning

§  Pipeline Partition

§  Dynamic Partition

§  Pushdown optimization

·         Preparation of test Cases

·         Executing Test case

·         Preparing Sample Data

·         Data Validation in source and target

·         Load And Performance testing

·         Unit testing Procedures

·         Error handling procedures

Contact us