How to Load or import data from HDFS to HBASE?

This is for Hadoop eco system like HDFS, Map reduce, Hive, Hbase, Pig, sqoop,sqoop2, Avro, solr, hcatalog, impala, Oozie, Zoo Keeper and Hadoop distribution like Cloudera, Hortonwork etc.
alpeshviranik
Posts: 81
Joined: Thu Jul 17, 2014 4:58 pm
Contact:

How to Load or import data from HDFS to HBASE?

Postby alpeshviranik » Wed Jul 23, 2014 12:20 am

I have installed Hadoop and HBase in my local. I have one file in HDFS and i want to import it in HBase.
how to Import data from HDFS to HBase? How i can import txt file from HDFS to HBase?


Guest

Re: How to Load or import data from HDFS to HBASE?

Postby Guest » Wed Jul 30, 2014 10:20 pm

You can use Pig to load data from HDFS to HBase because it is simple, straightforward, and flexible.
To load/import txt file in HBase from Hadoop.
    1) create HBase Table:
    $ hbase shell
    > create 'dataTable', 'mycfa'
    2) Move the text file to HDFS from Local:
    $ hadoop fs -put /home/Yourfile.txt /user/surendhar/HDFSfile.txt
    3) Then, write the pig script to store with HBaseStorage:
    A = LOAD 'HDFSfile.txt' USING PigStorage(',') as (strdata:chararray, intdata:long);
    STORE A INTO 'hbase://dataTable'
    USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
    'mycfa:intdata');
To load/import csv file from Hortonwork Example:
    1) load csv file in hadoop HDFS file system:
    hadoop fs -put hadoopinputdata.csv /tmp/
    2) create HBase table
    CREATE TABLE
    Hbase_table (user_id STRING, user_c1 STRING, user_c2 STRING)
    STORED BY 'org.apache.hcatalog.hbase.HBaseHCatStorageHandler'
    TBLPROPERTIES (
    'hbase.table.name' = 'simple_hcat_load_table',
    'hbase.columns.mapping' = 'da:c1,da:c2',
    'hcat.hbase.output.bulkMode' = 'true');
    3) Pig script to load csv file:
    A = LOAD 'hdfs:///tmp/hadoopinputdata.csv.tsv' USING PigStorage('\t') AS (user_id:chararray, user_c1:chararray, user_c2:chararray);
    -- DUMP A;
    STORE A INTO 'Hbase_table' USING org.apache.hcatalog.pig.HCatStorer();


Return to “Hadoop and Big Data”

Who is online

Users browsing this forum: No registered users and 8 guests