Tagarchief: Tools

Data & Analytics Testing Tools Summary (2)

Testing of Data & Analytics environments is increasingly supported by test tools. At Verified, we wonder what your experience is with one of these tools – and if you propose certain tools that have helped you ?

Please let us know, by leaving a message under this post, through the Contact FormLinkedIn or Twitter or contact us in any way you prefer. If you like, we can reply with a listing of more than 15 tools that should help testers and analysts  in testing and quality issues. 

The previous post on Test Tools can be found here.

Note: We are not sponsored in any way by the tool providers.

RTTS QuerySurge

Querysurge is a commercial product made for testing ETL, where comparison of source and target tables is the main object.

It offers dashboards, query wizards, various prefabricated queries for comparison of data, text search etc.

A trial version is available: https://www.querysurge.com/compare-trial-options

Note that for BI Testing, an add on is available. Connectors are available for Cognos, Tableau, SAP Business Objects, Microstrategy and OBIEE.  Features include comparison of reports when upgrading or migrating, querying report meta data.

ICEDQ

iCEDQ is a commercial ETL Test Tool, which runs in the cloud, on premise or hybrid.

It offers a rule engine module, which features:

  • Checksum rules (e.g. row counts)
  • Reconciliation rules (e.g. missing records)
  • Validity rules (SCD Type 2, Format Checks)
  • Script rules (e.g. execute DML/DDL, pre and post actions).

iCEDQ currently integrates with Jenkins, TFS, Jira, HP ALM and ServiceNow.

iCEDQ offers trials through their website:

https://icedq.com/download-icedq-trial

Bitwise QualiDI

QualiDI focuses on ETL testing from sources to targets.

It provides data validation and comparison of data, has a centralized repository of requirements, test cases, test results and has a API trigger based automation of test execution.

There is Business Rules Engine, enabling non-technical users to write rules instead of queries. It integrates with Bitwise Test Data Management Suite for providing test data.

QualiDi is a commercial product, a 30 day trial is available

Source: https://www.bitwiseglobal.com/innovations/qualidi/

SSIS Tester

SSIS Tester is offered by bytesoftwo and focuses on testing Microsoft SSIS.

SSIS Tester is a testing framework built on top of SQL Server Integration Services. It enables you to test packages, tasks and precedence constraints. It supports two types of tests:

  1. unit tests
  2. integration tests.

SSIS Tester helps you to develop you ETL process in the test driven manner and to identify errors early in the development process. It offers “data taps” and fake source or targe components, enabling developers to work independently. Tests target packages or particular tasks.

More information on SSIS Tester on MSDN can be found here:

http://msdn.microsoft.com/en-us/magazine/dn342874.aspx

SSIS Tester offers a free 21-day trial version:

http://www.bytesoftwo.com/download.html

Next 4 Tools

In a few weeks I will publish the next Blog post, asking for new experiences. Thanks for reading this post!

Omdat deze tekst gericht is op een internationaal publiek, heb ik ervoor gekozen om de bovenstaande tekst in het Engels op te stellen. Stelt u prijs op een vertaling naar het Nederlands? Neemt u dan aub contact met ons op.

Data & Analytics Testing Tools Summary (1)

Introduction

During the training Certified Data & Analytics Tester (Foundation Level) we provide a list of tools that can be used in testing or monitoring Business Intelligence and Data Warehouse products.

Below you can find the first 4 tools of this list, with a short description and links to the tool providers.

We are curious for your experiences when using these tools. What have they brought to your (testing) team? What is a Big Plus of using the tool – and with which features are you less impressed? Is an active user community available, or a help desk that makes a difference? Please let us know, by leaving a message under this post, through the Contact Form, LinkedIn or Twitter.

Note: We are not being sponsored by one of the products or their integration partners.

Data Cleaner

Data Cleaner is software which is used for Data Profiling.

It can also be used in the process of creating logical and physical test cases as it offers extensive support to analyze and browse through data sets.

Data Cleaner also offers features to clean, transform and export data sets. Rules can be built and reused.

Data Cleaner is free, additional (paid) features are available.

It can be downloaded to your desktop (Windows, Mac) environment from sourceforge or the community website:

https://datacleaner.org/

SAS Unit

SAS Unit is a framework for the unit testing of SAS programs (SAS EG 9.2, 9.3, 9.4).

Amongst others, SAS Unit offers features to assess test coverage of SAS Macros, running tests in batch mode, performance testing of SAS programs.

SAS Unit uses assertions to test the values of macro variables, contents of SAS data sets, relations between data sets, the existence of files incl. log files and – messages.

SAS Unit is open source software and built from SAS macros.

https://sourceforge.net/projects/sasunit/

Informatica DVO

Informatica offers a testing solution called Data Validation Option for PowerCenter (DVO).

The DataValidator for PowerCenter product was originally developed by a company DVO SOFTWARE. It is now available under the Informatica brand as Informatica PowerCenter Data Validation Option.

DVO is a custom tool built on top of Informatica PowerCenter. DVO integrates with the Informatica PowerCenter Repository and Integration Services and enables developers and business analysts to create rules to test the data being transformed during the data integration process.

Informatica mentions the following ETL Testing Use Cases: Production Validation Testing, Source to Target Testing and Application upgrades.

Amongst others, it features:

  • An easy-to-use GUI interface to test the rules created for data validations for multiple projects.
  • No programming skills needed to create validation tests.
  • A repository with reporting capabilities to provide a complete audit trail of all tests and their results.
  • Reading data definitions from PowerCenter metadata repositories and can easily deal with data definition changes.

Note that DVO tests data only, not mappings or workflows. Testing mappings is unit testing, which is different from data validation.

As DVO is a commercial product it may be attractive to Informatica Power Center users. At this moment, no free trial seems available (Feb 9, 2020).

www.informatica.com

Datagaps ETL Validator

Datagaps offers several products, e.g. ETL Validator, BI Validator, Data Flow and Test Data Manager.

The Datagaps ETL Validator comes pre-packaged with an ETL engine capable of extracting and comparing millions of records from multiple data sources while executing test cases in parallel.

ETL Validator also has a Visual Test Case Builder with drag-and-drop capabilities and a Query Builder that enables defining tests without manual typing.

Key features include enterprise collaboration, flat file testing, data profile testing, baselining, data quality testing, and database metadata testing.

Datagaps offers a free 30-day trial.

https://www.datagaps.com/etl-testing-tools/etl-validator-download

Next 4 Tools

The next page shows another 4 tools.

Omdat deze tekst gericht is op een internationaal publiek, heb ik ervoor gekozen om de bovenstaande tekst in het Engels op te stellen. Stelt u prijs op een vertaling naar het Nederlands? Neemt u dan aub contact met ons op.

Workshop “Data Quality” op Testnet Voorjaarsevenement 2016, Houten

Op woensdag 11 mei 2016 vindt het Voorjaarsevenement van Testnet plaats. Van 09:00-21:00 uur zal het NBC gevuld zijn met allerhande soorten testers, voor het delen van ervaringen en het opsnuiven van nieuwe kennis.

Armando Dörsek zal op deze dag met Peter Endema een workshop verzorgen: Data Profiling voor Testers.

De workshop vindt plaats tussen 15:40 uur en 17:20 uur. Neem je eigen laptop mee!

Na enige uitleg door Armando over oorzaken en gevolgen van data issues  neemt Peter de deelnemers mee in het geautomatiseerd analyseren van datasets middels een open source data profiling tool. Testers leren hiermee nieuwe vaardigheden waarmee ze eerder in het traject betrokken zullen worden en die hen nog waardevollere teamleden maakt.

Voor meer informatie over Testnet en het Voorjaarsevenement: https://www.testnet.org/testnet/p000533/bibliotheek/evenementen/voorjaarsevenement-2016-verbreed-je-basis-nieuwe-vaardigheden-voor-testers-korte-samenvatting-workshops