Skip to content

Hadoop core jar is missing in Hadoop 3

Issue –

Hadoop core jar is missing in Hadoop 3.

Environment –

  • Hadoop 2 or higher
  • Cloudera or Hortonworks distribution
  • Any application using Hive jdbc

Root cause –

From Cloudera repositories, hadoop-core jar is deprecated after version 2.6.0-mr1-cdh5.16.1 and from central repository hadoop-core is deprecated after version 1.2.1

If you are upgrading Hive/Hadoop your application in below versions then you may find hadoop-core missing.

  • Hadoop 1 to Hadoop 2 where Apache jars are used.
  • Hadoop 2 to Hadoop 3 where Cloudera jars are used.

Solution –

If you are using Apache jars then remove hadoop-core and add below dependencies.

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-mapreduce-client-core</artifactId>
    <version>3.3.6</version>
</dependency>

If you are using Cloudera jars then remove hadoop-core and add below dependencies.

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-mapreduce-client-core</artifactId>
    <version>3.1.1.7.2.14.0-149</version>
</dependency>

Also if you faced any noClassDefFound error after this then add hadoop-common dependency as well if missing.

For Apache –

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-common</artifactId>
    <version>3.3.6</version>
</dependency>

For Cloudera –

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-common</artifactId>
    <version>3.1.1.7.2.14.0-149</version>
</dependency>

Do you have another solution?

The solution provided above is based on the scenario our one of the developers/contributors faced. If you faced the same issue and found any other root cause then please share your solution in the comment section below. We will add the solution in this article.

Leave a Reply

Your email address will not be published. Required fields are marked *