duplicates in hive table

This is for Hadoop eco system like HDFS, Map reduce, Hive, Hbase, Pig, sqoop,sqoop2, Avro, solr, hcatalog, impala, Oozie, Zoo Keeper and Hadoop distribution like Cloudera, Hortonwork etc.
Site Admin
Posts: 186
Joined: Wed Jul 16, 2014 9:22 pm

duplicates in hive table

Postby forum_admin » Wed Jun 10, 2020 8:14 pm

How to check duplicates in hive table used on one or two columns?

e.g. table1 and check duplicates for col1

Posts: 44
Joined: Sat Jan 27, 2018 3:55 am

Re: duplicates in hive table

Postby ram123 » Wed Jun 10, 2020 8:16 pm

I tried it in HWT 2.6 and it is working.

Code: Select all

create table table2 as
Select A.* from
(select *,
row_Number() over (partition By col1 order by col1 desc) as row_num
from table1 )A
where row_num=1;

Return to “Hadoop and Big Data”

Who is online

Users browsing this forum: No registered users and 4 guests