What is Counters in Hadoop?

This is for Hadoop eco system like HDFS, Map reduce, Hive, Hbase, Pig, sqoop,sqoop2, Avro, solr, hcatalog, impala, Oozie, Zoo Keeper and Hadoop distribution like Cloudera, Hortonwork etc.
dharama123
Posts: 125
Joined: Wed Aug 27, 2014 1:10 am
Contact:

What is Counters in Hadoop?

Postby dharama123 » Thu Sep 18, 2014 2:48 am

What is Counters in Hadoop? Give an example and scenario where Counters use in Hadoop?


Guest

Re: What is Counters in Hadoop?

Postby Guest » Sat Sep 20, 2014 10:07 pm

Sometimes when running MapReduce jobs, you want to know whether or how often a certain event has occured during execution. Imagine an iterative algorithm that should run until no changes were made to the data during execution.

public class Counters extends AbstractCounters<Counters.Counter,Counters.Group>

A set of named counters.

Counters represent global counters, defined either by the Map-Reduce framework or applications. Each Counter can be of any Enum type.
Counters are bunched into Counters.Groups, each comprising of counters from a particular Enum class.


Hadoop maintains some built-in counters for every job, which report various metrics for your job.
For example, there are counters for the number of bytes and records processed, which allows you to confirm that the expected amount of input was consumed and the expected amount of output was produced.

counter identify by below method:
context.getCounter(MyCounters.Counter).increment(1);
job.getCounters().findCounter(MyCounters.Counter).getValue();


Return to “Hadoop and Big Data”

Who is online

Users browsing this forum: No registered users and 1 guest