- This topic has 1 reply, 1 voice, and was last updated 5 years, 7 months ago by
.
-
Posts
-
-
-
Sqoop Validation
A simple definition of Sqoop validation validates the data copied. By comparing the row counts from the source as well as the target post copy, either import or Export .Especially, to compare the row counts between source as well as the target just after data imported into HDFS, we use this option.
All the rows are deleted or added, while during the imports, though, Sqoop tracks this change. Moreover, it updates the log file.
Interfaces of Sqoop Validation
3 interfaces of Sqoop Validation are:a. ValidationThreshold
In order to determine whether the error margin between the source and target are acceptable(Absolute, Percentage Tolerant and many more) we use the ValidationThreshold. Although, its default implementation is AbsoluteValidationThreshold.Basically, that ensures that the row counts from source as well as targets are the same.
b. ValidationFailureHandler
Also, it has once interface with ValidationFailureHandler, that is responsible for handling failures here. Such as log an error/warning, abort and many more. Although default implementation is LogOnFailureHandler. Here that logs a warning message to the configured logger.c. Validator
Generally, Validator drives the validation logic, by delegating the decision to ValidationThreshold.However to learn about Sqoop validation in detail, follow the link: Sqoop Validation – Interfaces & Limitations of Sqoop Validate
-
- You must be logged in to reply to this topic.
