Search found 61 matches

by snehalshah
Wed Sep 30, 2015 11:43 pm
Forum: Hadoop and Big Data
Topic: how to Configure Kerberos Authentication in Hortonworks Hadoop HDP 2.2?
Replies: 2
Views: 777

Re: how to Configure Kerberos Authentication in Hortonworks Hadoop HDP 2.2?

Quick and short Step by Step guide to configure kerberos authentication in hortonworks hadoop HDP 2.2 by Kuldeep Kulkarni on

http://crazyadmins.com
by snehalshah
Wed Sep 30, 2015 11:41 pm
Forum: Hadoop and Big Data
Topic: migrate data Using flume from mysql to Cassandra
Replies: 1
Views: 561

Re: migrate data Using flume from mysql to Cassandra

Created a mysql source and cassandra sink to migrate data from mysql to Cassandra. you can do it by following steps.

https://github.com/sksumit1/bigdata/tree/master/flume
by snehalshah
Wed Sep 30, 2015 11:41 pm
Forum: Hadoop and Big Data
Topic: Map reduce total word count help
Replies: 1
Views: 531

Re: Map reduce total word count help

I think this tutorial will help you: https://www.youtube.com/watch?v=a_-Djndi8ZE
by snehalshah
Wed Sep 30, 2015 11:36 pm
Forum: Hadoop and Big Data
Topic: Job scheduling in Hadoop for Heterogeneous cluster
Replies: 1
Views: 474

Re: Job scheduling in Hadoop for Heterogeneous cluster

Use oozie workflow. It provides you way to get callbacks when the workflow enters / exits your job. You can have a start-stop timer to calculate the run time.
by snehalshah
Wed Sep 30, 2015 11:35 pm
Forum: Hadoop and Big Data
Topic: upgrade Hadoop-1.2.1 to Hadoop-2.6.0
Replies: 1
Views: 484

Re: upgrade Hadoop-1.2.1 to Hadoop-2.6.0

If you have data then there steps to migrate from non-YARN to YARN. like take the backup of namenode and do the upgrade for required hadoop folder.

in case there is no data than remove the hadoop folder with new one.
just stop the older cluster and start the newer one, wouldn't make much difference.
by snehalshah
Wed Sep 30, 2015 11:32 pm
Forum: Hadoop and Big Data
Topic: error "Name node is in safe mode"
Replies: 1
Views: 504

Re: error "Name node is in safe mode"

file of the size would not matter. just check the block size and other configuration. you can try with below commands: hdfs dfsadmin -safemode leave sudo -u hdfs hdfs dfsadmin -safemode leave If you wish instances to be launched on all the nodes, in your case three, set your replication factor accor...
by snehalshah
Wed Sep 30, 2015 11:29 pm
Forum: Hadoop and Big Data
Topic: MS thesis on Big Data in health care and medical applications
Replies: 0
Views: 1339

MS thesis on Big Data in health care and medical applications

Hi all,

I am searching to find a good topic for my MS thesis on Big Data in health care and medical applications.
Any idea? I will appreciate to receive a good problem.
Thanks,
by snehalshah
Wed Sep 30, 2015 11:27 pm
Forum: Hadoop and Big Data
Topic: NLP API or Utility for Hadoop
Replies: 0
Views: 1305

NLP API or Utility for Hadoop

looking for NLP API or Utility for Hadoop. I am working on large scare text based analysis. More precisely I am doing Sentiment analysis on Twitter data for particular products. I am using Flume to pull Twitter data in HDFS. Is there any NLP API or Utility I can apply on these twitts to get correct ...
by snehalshah
Wed Sep 30, 2015 11:26 pm
Forum: Hadoop and Big Data
Topic: how to Load XML data into HBase using MapReduce?
Replies: 0
Views: 1418

how to Load XML data into HBase using MapReduce?

How to do Bulk Loading XML data into HBase using MapReduce?
by snehalshah
Wed Sep 30, 2015 11:25 pm
Forum: Hadoop and Big Data
Topic: predict an application execution time in Hadoop Yarn?
Replies: 0
Views: 1308

predict an application execution time in Hadoop Yarn?

is it possible to predict an application execution time, that is submitted to Hadoop cluster or the resource requirements of the same?

how to predict an application execution time in Hadoop Yarn?
by snehalshah
Tue Sep 29, 2015 11:08 pm
Forum: Bio-Technology and Biology
Topic: what is the Hematology?
Replies: 1
Views: 491

what is the Hematology?

what is the Hematology? How it is differ than biology?
by snehalshah
Tue Sep 29, 2015 11:06 pm
Forum: Objective C and Cocoa Touch Framework for iOS Apps
Topic: component of Cocoa Touch Framework
Replies: 1
Views: 3278

component of Cocoa Touch Framework

What is the Cocoa Touch Framework for iOS Apps? Which are the component of Cocoa Touch Framework?
by snehalshah
Tue Sep 29, 2015 11:05 pm
Forum: Bio-Technology and Biology
Topic: Technology to study human cell
Replies: 1
Views: 503

Technology to study human cell

How one can study human cell? which are the technology and tools available to study the human cells?
by snehalshah
Tue Sep 29, 2015 11:04 pm
Forum: Bio-Technology and Biology
Topic: BioTechnology vs Biology
Replies: 1
Views: 542

BioTechnology vs Biology

It seems Bio-Technology and Biology are the same. Why there are two different streams for study?
What is the difference between Bio-Technology and Biology?
by snehalshah
Tue Sep 29, 2015 11:02 pm
Forum: Yoga, Exercise, Pilates and Zim
Topic: Who is baba ramdev? why he is so famous in India?
Replies: 1
Views: 568

Who is baba ramdev? why he is so famous in India?

Who is baba ramdev? why he is so famous in India?

can anyone give any feedback on his life and goal?
by snehalshah
Tue Sep 29, 2015 11:01 pm
Forum: Yoga, Exercise, Pilates and Zim
Topic: What are the types of Pilates?
Replies: 1
Views: 609

What are the types of Pilates?

Which are the different types of Pilates?
by snehalshah
Tue Sep 29, 2015 11:01 pm
Forum: Yoga, Exercise, Pilates and Zim
Topic: Pilates vs Yoga
Replies: 1
Views: 583

Pilates vs Yoga

What is the difference between pilates and Yoga? Which one is best for good health?
by snehalshah
Tue Sep 29, 2015 11:00 pm
Forum: Yoga, Exercise, Pilates and Zim
Topic: What is Pilates?
Replies: 0
Views: 1256

What is Pilates?

What is Pilates? What is the advantage of Pilates?
if any one have experience to do pilates, than please share the experience.
by snehalshah
Tue Sep 29, 2015 10:59 pm
Forum: Religious Debates
Topic: Who is Mahakali? Why she began to kill every one?
Replies: 0
Views: 1288

Who is Mahakali? Why she began to kill every one?

Who is Mahakali? Why she began to kill every one?
by snehalshah
Tue Sep 29, 2015 10:57 pm
Forum: Religious Debates
Topic: Who is the god for Jain community?
Replies: 1
Views: 2879

Who is the god for Jain community?

Who is the god for Jain community? how they are different than Hindu religious?
by snehalshah
Tue Sep 29, 2015 10:56 pm
Forum: Religious Debates
Topic: Who wrote Bhagvad Gita? Why?
Replies: 1
Views: 568

Who wrote Bhagvad Gita? Why?

Who wrote Bhagvad Gita? Why?
by snehalshah
Tue Sep 29, 2015 10:55 pm
Forum: Religious Debates
Topic: Who wrote Bible? When and how it distributed?
Replies: 0
Views: 1311

Who wrote Bible? When and how it distributed?

Who wrote Bible? When and how it distributed?
by snehalshah
Tue Sep 29, 2015 10:54 pm
Forum: Religious Debates
Topic: Who started the islam? When islam started?
Replies: 0
Views: 1337

Who started the islam? When islam started?

Who started the islam? When islam started?
by snehalshah
Wed Sep 23, 2015 8:53 pm
Forum: Core Java, Scala, Python and Advanced Programming or development
Topic: how to add library in eclipse project?
Replies: 2
Views: 772

Re: how to add library in eclipse project?

You can add it by more than one way. 1.Start Eclipse, and locate the project folder to which this ACM library should be added. 2.Right-click this class folder, and select "Properties" 3.Select "Java Build Path" on the left, and then the "Libraries" tab. and click on add...
by snehalshah
Wed Sep 23, 2015 8:34 pm
Forum: Core Java, Scala, Python and Advanced Programming or development
Topic: how to add library in eclipse project?
Replies: 2
Views: 772

how to add library in eclipse project?

how to add library in eclipse project? I want to use stanfordnlp library in my java project.

how can I add or import library in eclipse project?
by snehalshah
Wed Sep 23, 2015 8:21 pm
Forum: Entrance Exams after 12th, Graduation and for admission outside and inside the India
Topic: GATE preparation for BE, BTech student
Replies: 2
Views: 461

GATE preparation for BE, BTech student

when should I start GATE preparation? When I can appear for GATE exam?
I am 2nd year BE student.
by snehalshah
Wed Sep 23, 2015 8:20 pm
Forum: health and food
Topic: avacado paratha
Replies: 0
Views: 1273

avacado paratha

is avocado paratha have same protein as normal avocado have? My query is that protein will burn if we will fry avocado paratha?
by snehalshah
Wed Sep 23, 2015 8:17 pm
Forum: health and food
Topic: kids have obesity problem in USA
Replies: 0
Views: 1264

kids have obesity problem in USA

kids in USA getting many kinds of food an they got overloaded with food.

is there any procedure to aware all children about obesity issue in USA?
by snehalshah
Wed Sep 23, 2015 8:15 pm
Forum: health and food
Topic: How to control on diet
Replies: 0
Views: 1278

How to control on diet

I am very foodie and can't control my eating habit.

Is there any way to control my eating habit?
by snehalshah
Wed Sep 23, 2015 7:23 pm
Forum: Hadoop and Big Data
Topic: java.io.IOException: Could not load jar /usr/lib/sqoop-connector-teradata-1.4c5/sqoop-connector-teradata-1.4c5.jar into JVM
Replies: 1
Views: 887

Re: java.io.IOException: Could not load jar /usr/lib/sqoop-connector-teradata-1.4c5/sqoop-connector-teradata-1.4c5.jar into JVM

download Teradata connector powered by Teradata or cloudera. untar it in /usr/lib/ drirectory.
somehow you have configure the connector path in /etc/lib/sqoop/conf/managers.d/cldra_td_connector file
by snehalshah
Wed Sep 23, 2015 6:38 pm
Forum: health and food
Topic: which kind of Foods your kid should avoid?
Replies: 1
Views: 461

Re: which kind of Foods your kid should avoid?

Below food you can avoid for your kid. I know below things are common as kids food in Costco and walmart.

Granola Bars
Pizza, noodles and pasta
flavored yogurt
fruit snacks
Smoothies
by snehalshah
Wed Sep 23, 2015 5:19 pm
Forum: Hadoop and Big Data
Topic: install hadoop in Pseudo-Distributed Mode
Replies: 1
Views: 623

Re: install hadoop in Pseudo-Distributed Mode

download jar from below site. http://hadoop.apache.org/releases.html start install: root@root-hadoop-virtual-machine:~$ tar xvfz hadoop-2.3.0.tar.gz root@root-hadoop-virtual-machine:~$ cd hadoop-2.3.0/ root@root-hadoop-virtual-machine:~/hadoop-2.3.0$ ls bin include libexec NOTICE.txt sbin etc lib LI...
by snehalshah
Wed Sep 23, 2015 5:08 pm
Forum: Hadoop and Big Data
Topic: rm and rmr command in Hadoop File System
Replies: 1
Views: 984

Re: rm and rmr command in Hadoop File System

for file you should use rm
hadoop fs -rm a.txt

for dir you should use -rm -r -f option
hadoop fs -rm -r -f a_dir
by snehalshah
Wed Sep 23, 2015 5:07 pm
Forum: Hadoop and Big Data
Topic: fs command in Hadoop
Replies: 1
Views: 653

Re: fs command in Hadoop

fs will access hdfs file system. so use it before any file system command like copy, merge, ls etc...

hadoop fs -put a.txt a.txt
by snehalshah
Wed Sep 23, 2015 5:05 pm
Forum: Hadoop and Big Data
Topic: getmerge command in Hadoop File System
Replies: 3
Views: 719

Re: getmerge command in Hadoop File System

hadoop fs -getmerge /input_dir output.file
by snehalshah
Wed Sep 23, 2015 5:03 pm
Forum: Hadoop and Big Data
Topic: distcp command in hadoop
Replies: 1
Views: 598

Re: distcp command in hadoop

hadoop fs distcp a.txt b.txt
by snehalshah
Wed Sep 23, 2015 5:01 pm
Forum: Hadoop and Big Data
Topic: jar command in Hadoop
Replies: 1
Views: 609

Re: jar command in Hadoop

hadoop jar test.jar test input output
by snehalshah
Wed Sep 23, 2015 4:59 pm
Forum: Hadoop and Big Data
Topic: queue command in Hadoop
Replies: 1
Views: 606

Re: queue command in Hadoop

hadoop [ Generic Options ] queue [-list] | [-info <job-queue-name> [-showJobs]] | [-showacls]
by snehalshah
Wed Sep 23, 2015 4:58 pm
Forum: Hadoop and Big Data
Topic: pipes command in Hadoop
Replies: 2
Views: 622

Re: pipes command in Hadoop

hadoop fs -cat test.txt | hadoop fs get test.txt
by snehalshah
Wed Sep 23, 2015 4:56 pm
Forum: Hadoop and Big Data
Topic: version,CLASSNAME, classpath command in Hadoop
Replies: 1
Views: 519

Re: version,CLASSNAME, classpath command in Hadoop

hadoop version

hadoop -cp `hadoop classpath` test.jar testclass input output
by snehalshah
Tue Sep 22, 2015 11:05 pm
Forum: Hadoop and Big Data
Topic: under replicated blocks in cloudera hadoop cluster
Replies: 0
Views: 1354

under replicated blocks in cloudera hadoop cluster

I am using the replication factor 2. Save configuration, Deploy Client Configuration, steps done. The configuration shows it as 2 and on nodes, the xml files as well. But, when I add files/run mapred jobs, the default of 3 is enforced. Where else is to modify replication factor ? LOG: Health Tests(C...
by snehalshah
Tue Sep 22, 2015 11:00 pm
Forum: Hadoop and Big Data
Topic: Error: com.teradata.hadoop.exception.TeradataHadoopException: com.teradata.jdbc.jdbc_4.util.JDBCException
Replies: 0
Views: 1328

Error: com.teradata.hadoop.exception.TeradataHadoopException: com.teradata.jdbc.jdbc_4.util.JDBCException

Sqoop export to Teradata failed with timeout. INFO mapreduce.Job: map 0% reduce 0% 14/11/17 14:35:05 INFO mapreduce.Job: Task Id : attempt_1415914341678_2799_m_000000_0, Status : FAILED Error: com.teradata.hadoop.exception.TeradataHadoopException: com.teradata.jdbc.jdbc_4.util.JDBCException: [Terada...
by snehalshah
Tue Sep 22, 2015 10:59 pm
Forum: Hadoop and Big Data
Topic: java.sql.SQLException: [Simba][HiveJDBCDriver](500310) Invalid operation: Unable to obtain Princpal Name for authenticat
Replies: 0
Views: 1405

java.sql.SQLException: [Simba][HiveJDBCDriver](500310) Invalid operation: Unable to obtain Princpal Name for authenticat

Error during connectionto Hive server with Cloudera JDBC driver Exception in thread "main" java.sql.SQLException: [Simba][HiveJDBCDriver](500310) Invalid operation: Unable to obtain Princpal Name for authentication ; at com.cloudera.hive.hivecommon.api.HiveServer2ClientFactory.createTransp...
by snehalshah
Tue Sep 22, 2015 10:57 pm
Forum: Hadoop and Big Data
Topic: Could not load ERROR sqoop.Sqoop: db driver class: net.sourceforge.jtds.jdbc.Driver
Replies: 0
Views: 1448

Could not load ERROR sqoop.Sqoop: db driver class: net.sourceforge.jtds.jdbc.Driver

sqoop command fails. ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: net.sourceforge.jtds.jdbc.Driver java.lang.RuntimeException: Could not load db driver class: net.sourceforge.jtds.jdbc.Driver at org.apache.sqoop.manager.SqlManager.makeCo...
by snehalshah
Tue Sep 22, 2015 10:56 pm
Forum: Hadoop and Big Data
Topic: java.io.IOException: Header doesn't contain versioninfo
Replies: 1
Views: 2329

Re: java.io.IOException: Header doesn't contain versioninfo

Check that pdfbox jar is available in all node. I think it is not able to find pdfbox jar file one of the node.
by snehalshah
Tue Sep 22, 2015 10:54 pm
Forum: Hadoop and Big Data
Topic: put: unexpected URISyntaxException
Replies: 1
Views: 1912

Re: put: unexpected URISyntaxException

I think space is the issue. so, just use below command.

[root@userram] hadoop fs -put hadoop%20input.text hadoop%20input.text
by snehalshah
Tue Sep 22, 2015 10:53 pm
Forum: Hadoop and Big Data
Topic: "No suitable driver found for jdbc:derby" error for sqoop2 upgrade
Replies: 1
Views: 998

Re: "No suitable driver found for jdbc:derby" error for sqoop2 upgrade

check below thing first: Copy your derby jar from /opt/cloudera/parcels/CDH/jars/derby-10.8.2.2.jar to /var/lib/sqoop2 Set owner as sqoop owned by sqoop2:sqoop. Then run the Cloudera Manager > Services > Sqoop2 > Actions > Upgrade Sqoop. Once the upgrade is done, remove the /var/lib/sqoop2/derby-10....
by snehalshah
Tue Sep 22, 2015 10:50 pm
Forum: Hadoop and Big Data
Topic: java.sql.SQLException: [Teradata Database] [TeraJDBC 15.00.00.28] [Error 8017] [SQLState 28000] The UserId, Password or Account is invalid
Replies: 1
Views: 726

Re: java.sql.SQLException: [Teradata Database] [TeraJDBC 15.00.00.28] [Error 8017] [SQLState 28000] The UserId, Password

check your userid and password is working for Teradata. you can also check for logmech option as per the company policy.

jdbc:teradata://whlsp.wellsfargo.com/DATABASE=TEST_DB,logmech=ldap
by snehalshah
Tue Sep 22, 2015 10:49 pm
Forum: Hadoop and Big Data
Topic: sqoop export Error: com.teradata.connector.common.exception.ConnectorException: index outof boundary
Replies: 1
Views: 2528

Re: sqoop export Error: com.teradata.connector.common.exception.ConnectorException: index outof boundary

sqoop export --connect jdbc:teradata://localhost.com/DATABASE=DBNAME,logmech=ldap --username u123 --password test1407 --export-dir /user/root/out_MR_proddata --table table_product --num-mappers 3 --input-fields-terminated-by ';' above command is correct just check the input file have exact column as...
by snehalshah
Tue Sep 22, 2015 10:45 pm
Forum: Hadoop and Big Data
Topic: Caused by: java.sql.SQLException: [Teradata JDBC Driver] [TeraJDBC 15.00.00.28] [Error 1000] [SQLState 08S01] Login failure for Connection to localhost
Replies: 1
Views: 834

Re: Caused by: java.sql.SQLException: [Teradata JDBC Driver] [TeraJDBC 15.00.00.28] [Error 1000] [SQLState 08S01] Login

The issue is on schema and it is common one. Check the table columns in tearadata and your export file.

Also check the comma in your data. in that case you can use ;,; as field separator.
by snehalshah
Fri Sep 18, 2015 9:25 pm
Forum: Hadoop and Big Data
Topic: org.apache.hadoop.fs.ChecksumException: Checksum error: file:/root/out_merge.txt at 0 exp: -289599598 got: 1159585454
Replies: 1
Views: 1188

org.apache.hadoop.fs.ChecksumException: Checksum error: file:/root/out_merge.txt at 0 exp: -289599598 got: 1159585454

hadoop put command giving checksum error [root@localhost ~]# hadoop fs -put out_merge.txt 15/09/18 17:23:22 INFO fs.FSInputChecker: Found checksum error: b[0, 327]=41677265656d656e7420446174653a204e6f76656d62657220312c203230313109303132353339363436380a5072696e636970616c20416d6f756e743a202824312c3234...
by snehalshah
Fri Sep 18, 2015 9:05 pm
Forum: Hadoop and Big Data
Topic: sqoop export Error: com.teradata.connector.common.exception.ConnectorException: index outof boundary
Replies: 1
Views: 2528

sqoop export Error: com.teradata.connector.common.exception.ConnectorException: index outof boundary

I am getting below error during export file from hdfs to Teradata user@root] sqoop export --connect jdbc:teradata://localhost.com/DATABASE=DL_FMRA,logmech=ldap --username u123 --password test1407 --export-dir /user/root/out_MR_proddata --table table_product --num-mappers 3 --input-fields-terminated-...
by snehalshah
Thu Sep 17, 2015 9:18 pm
Forum: Hadoop and Big Data
Topic: Caused by: java.sql.SQLException: [Teradata JDBC Driver] [TeraJDBC 15.00.00.28] [Error 1000] [SQLState 08S01] Login failure for Connection to localhost
Replies: 1
Views: 834

Caused by: java.sql.SQLException: [Teradata JDBC Driver] [TeraJDBC 15.00.00.28] [Error 1000] [SQLState 08S01] Login failure for Connection to localhost

I am trying to export hdfs data in Teradata but getting connection error. user@root]sqoop export --connect jdbc:teradata://localhost/TEST,logmech=ldap --username test --password test --export-dir HDFS_dir --table hdfs_dir_table --num-mappers 3 --fields-terminated-by ',' 2015-09-17.17:11:40.557 TERAJ...
by snehalshah
Thu Sep 17, 2015 9:02 pm
Forum: Hadoop and Big Data
Topic: Error: java.lang.ArrayIndexOutOfBoundsException: 7 at com.teradata.connector.hdfs.serde.HdfsTextSerDe.deserialize(HdfsTextSerDe.java:293)
Replies: 0
Views: 1533

Error: java.lang.ArrayIndexOutOfBoundsException: 7 at com.teradata.connector.hdfs.serde.HdfsTextSerDe.deserialize(HdfsTextSerDe.java:293)

export from hdfs to Teradata is failing. user@root]sqoop export --connect jdbc:teradata://localhost/DATABASE=TEST,logmech=ldap --username test123 --password test123 --export-dir hdfs_MR_Dir --table hdfs_MR_Table--num-mappers 3 --fields-terminated-by ',' --lines-terminated-by '\n' 15/09/17 16:57:22 I...
by snehalshah
Thu Sep 10, 2015 7:49 pm
Forum: Hadoop and Big Data
Topic: java.sql.SQLException: [Simba][HiveJDBCDriver](500310) Invalid operation: Unable to obtain Princpal Name for authenticat
Replies: 6
Views: 3413

Re: java.sql.SQLException: [Simba][HiveJDBCDriver](500310) Invalid operation: Unable to obtain Princpal Name for authent

Print the Kerberos debug by adding the below parameter: -Dsun.security.krb5.debug=true -Dsun.security.jgss.debug=true you will get >>> unsupported key type found the default TGT: 18 Exception in thread "main" java.sql.SQLException: [Simba][ImpalaJDBCDriver](500310) Invalid operation: Unabl...
by snehalshah
Thu Sep 10, 2015 7:47 pm
Forum: Hadoop and Big Data
Topic: Error: com.teradata.hadoop.exception.TeradataHadoopException: com.teradata.jdbc.jdbc_4.util.JDBCException:
Replies: 1
Views: 414

Re: Error: com.teradata.hadoop.exception.TeradataHadoopException: com.teradata.jdbc.jdbc_4.util.JDBCException:

There are some new nodes which recently added to cluster. Teradata is not able to access those node.
You need to enable access from these new nodes to Teradata.


Othere way you can shutdown those nodes by Shutdown the NodeManager role on that Nodes. So, mapper will not run on those nodes.

Go to advanced search