When kafka delete message?

This is for Hadoop eco system like HDFS, Map reduce, Hive, Hbase, Pig, sqoop,sqoop2, Avro, solr, hcatalog, impala, Oozie, Zoo Keeper and Hadoop distribution like Cloudera, Hortonwork etc.
forum_admin
Site Admin
Posts: 185
Joined: Wed Jul 16, 2014 9:22 pm
Contact:

When kafka delete message?

Postby forum_admin » Sat Sep 24, 2016 9:13 pm

When kafka delete message?


Guest

Re: When kafka delete message?

Postby Guest » Sat Sep 24, 2016 9:42 pm

Once comsumer have processed the message, it will ack to brokere and broker will delete it.

There is retention log property that will delete the data after consume.

Guest

Re: When kafka delete message?

Postby Guest » Sat Sep 24, 2016 9:48 pm

Default time for the log is set for 168 hours and then the Data is automatically removed from the Kafka-Topic which you created.

go to the server.properties which is located in the config folder and the change the 168 to a minimum time. so their is no data after the specific amount of time which you have set for the log.retention.hours.So your issue will be solved.

log.retention.hours=100



Return to “Hadoop and Big Data”

Who is online

Users browsing this forum: No registered users and 1 guest