- This topic has 1 reply, 1 voice, and was last updated 5 years, 7 months ago by
.
-
Posts
-
-
-
Counters are used to keep track of occurrence of events.In hadoop framework whenever any mapreduce job gets executed then hadoop initiates counters to keep track track of different statistics like number of rows read or number of rows written.
There are generally tow types of counters:
1)Built in counters
2)Custom counters.Built in counters are of three types:
a) Map reduce Task counter:
Whenever a task gets executes all over the node then its results are aggregated all over the node.for example:
Number of records read by all the map task then final result is the total sum of records read by all the map task.
examples:MAP_INPUT_RECORDS,MAP_INPUT_BYTES etc
b) File System counters:
It keeps track of number of bytes read or written by the file system.
example:
BYTES_READ-number of bytes read by map task via FileInputFormat.
BYTES_WRITTEN.-number of bytes written by map or reduce task.c) Job Counters
They keep track of job level statistics like number of maps launched or number of reducers launched.
Custom counters:
This users defines according to their convenience to track for certain values in the input over execution of job.This counter is defined by ENUM in where all the fields are defined which has to be tracked.
example:
public enum track_values
{
value1,
value1
};Each time value1 or value1 is found in the input file,their counter gets incremented by one.
-
- You must be logged in to reply to this topic.
