Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

In this case SuperCHANNEL has used Batch Insertion, and all rows from source have been channelled to the target:

Note
iconfalse

It is important to check the logs after each build to confirm that the number of rows channelled for each table matches the number of rows in that table. If any rows are missing, for example: channelled 454282 rows (out of 454286) then this may indicate issues with your source data.

SuperCHANNEL also displays the total number of cleansing actions found during the run and indicates that the build is complete:

...

You can change the default increment by editing the str.progress.report.interval parameter in the SuperCHANNEL configuration file, config.txt .

Error Log

If there are errors in the build, they display on the Errors tab.

Image Added

In the following example there are two warning messages:

Code Block
languagetext
WARNING - Unable to load JDBC driver: str.jdbc.komodo.Driver
WARNING - Unable to load JDBC driver: str.jdbc.ddi.Driver 

The messages indicate that two drivers that were specified in the SuperCHANNEL configuration, config.txt , could not be found. Both of these are Space-Time Research drivers and were not required for this build:

  • str.jdbc.komodo is the Space-Time Research driver required for connection to the deprecated SXV3 format, and is only available in the 32-bit version of SuperCHANNEL.
  • str.jdbc.ddi is the driver for connection to data in DDI format. This driver is only available in an extended DDI version of SuperCHANNEL.

Error Messages

Target Schema validation failed: No CODE column defined for classification table

...

To resolve this issue, use the Target Attributes pane to specify which column in the classification table is the Name column:

...

SQLException

If there are errors in the build, they display on the Errors tab.

Image Removed

In the following example there are two warning messages:

Code Block
languagetext
WARNING - Unable to load JDBC driver: str.jdbc.komodo.Driver
WARNING - Unable to load JDBC driver: str.jdbc.ddi.Driver 

The messages indicate that two drivers that were specified in the SuperCHANNEL configuration, config.txt, could not be found. Both of these are Space-Time Research drivers and were not required for this build:

...

any SQLException error messages displayed in the Errors tab, then this indicates a serious data problem. In most cases this type of error is caused by issues with the source data.

For example:

Code Block
BatchDispatcher encountered SQLException: -- Could not look up compound keys ( 7772 ) in index Tables/F_Customer/Indexes/PrimaryKey while inserting data into table F_Account, column: ( FK_Cust_Key ), join: Joins/F_Account_20_F_Customer_0_Joins

In this example, there are two fact tables (F_Accounts and F_Customer) linked with a primary key/foreign key relationship. This error message indicates that the F_Account fact table has a record with a foreign key value of 7772 that could not be found in the primary key column of the F_Customer table. This either means that a record is missing from the F_Customer table in the source or that the record with that ID has been skipped for some reason.

When troubleshooting this issue, a good next step is to check the Progress logs for any issues. For example:

Code Block
Using Batch Insertion for table: F_Customer
--Action: skipping at column "Area" found invalid value "9090" on row 0...skipped 
-- channelled 269191 rows (out of 269192) successfully to "F_Customer"
--      0hrs 0min 3sec
--Started channelling from "F_Account"...
Using Batch Insertion for table: F_Account
-- channelled 0 rows (out of 19902) successfully to "F_Account"
--      0hrs 0min 0sec
--Commiting changes to target database
--0hrs 0min 4sec
120 cleansing action(s)

In this case we can see that one of the records in the F_Customer table has an invalid post code in the Area column. As this is a hierarchy, which does not support the add to classification cleansing action, the row has been skipped: channelled 269191 rows (out of 269192) successfully to "F_Customer",

As a result, SuperCHANNEL has been unable to link the two fact tables together, and has skipped the entire F_Account table: channelled 0 rows (out of 19902) successfully to "F_Account"

The solution here is to resolve the issue in the source data by doing one of the following:

  • If the value of Area in the F_Customer record in question is valid, add it to the classification tables (with a suitable foreign key to the next level in the hierarchy).
  • If the value of Area in the F_Customer record is not a valid postcode, change it to a valid value.