Hive FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException:

This is for Hadoop eco system like HDFS, Map reduce, Hive, Hbase, Pig, sqoop,sqoop2, Avro, solr, hcatalog, impala, Oozie, Zoo Keeper and Hadoop distribution like Cloudera, Hortonwork etc.
forum_admin
Site Admin
Posts: 179
Joined: Wed Jul 16, 2014 9:22 pm
Contact:

Hive FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException:

Postby forum_admin » Wed May 31, 2017 12:02 am

Hive is not running. giving Read timed out error.

[hadoop@localhost1 bin]$ hive
Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.4.3-1.cdh5.4.3.p0.6/jars/hive-common-1.1.0-cdh5.4.3.jar!/hive-log4j.properties
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out


forum_admin
Site Admin
Posts: 179
Joined: Wed Jul 16, 2014 9:22 pm
Contact:

Re: Hive FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException:

Postby forum_admin » Wed May 31, 2017 12:04 am

Meanwhile the steps I have taken for better performance and to avoid this problem are:
1. Database connection pooling tuning,
Default is 10, made it 30 on each thrift server.
Though the the DBCP Connection Pool ( maximum connections) config also need to be think though as that will also have implication of using MySQL resources.
2. JVM GC Tuning
3. keeping number of partitions in tact

Please check below link:
https://issues.apache.org/jira/browse/HIVE-9469

forum_admin
Site Admin
Posts: 179
Joined: Wed Jul 16, 2014 9:22 pm
Contact:

Re: Hive FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException:

Postby forum_admin » Wed May 31, 2017 12:05 am

it is heap error. look in log. it may shows below exit code:
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

Hive/HiveServer2/Configuration/Resource Management/Java Heap Size
Hive/Hive Metastore/Configuration/Resource Management/Java Heap Size

mapreduce.map.memory.mb
mapreduce.reduce.memory.mb
mapreduce.map.java.opts -> 80% of mapreduce.map.memory.mb
mapreduce.reduce.java.opts -> 80% of mapreduce.reduce.memory.mb



Return to “Hadoop and Big Data”

Who is online

Users browsing this forum: No registered users and 2 guests