how to import streaming data from Kafka into HBase

This is for Hadoop eco system like HDFS, Map reduce, Hive, Hbase, Pig, sqoop,sqoop2, Avro, solr, hcatalog, impala, Oozie, Zoo Keeper and Hadoop distribution like Cloudera, Hortonwork etc.
forum_admin
Site Admin
Posts: 185
Joined: Wed Jul 16, 2014 9:22 pm
Contact:

how to import streaming data from Kafka into HBase

Postby forum_admin » Sat Sep 24, 2016 9:16 pm

what are best practices for importing streaming data from Kafka into HBase?


hadoopdevloper

Re: how to import streaming data from Kafka into HBase

Postby hadoopdevloper » Sat Sep 24, 2016 9:37 pm

You can select any option

Kafka --> Flume --> HBase
Kafka --> Storm --> HBase
Kafka --> Spark Streaming --> HBase, Kafka --> HBase

it also depends on below questions:
1) What is the volume in terms of messages per second?
2) How big is an individual message
3) Are there other transformation or processing that happens other than the protobuf deserialization?
4) Are all messages the same or are there multiple message types and /or topics?
5) Are you running your Hbase cluster with Kerberos enabled?


Return to “Hadoop and Big Data”

Who is online

Users browsing this forum: No registered users and 1 guest