while starting hadoop services, datanode service is not running

In Hadoop, problem occurs on DataNode, so first goto log’s dir,
1. Cd logs
2. Ls
3. Less hadoop-ubuntu-datanode-uduntu.log

Here we can easily detect the problem, and our problem is incompatible clusterIDs.
java.io.IOException: Incompatible clusterIDs in /home/hdadmin/hdata2/dfs/data: namenode clusterID = CID-99237010-410d-412e-8ec3-a203bbe991ea; datanode clusterID = CID-a390f83a-76bd-4f08-92a7-5cac6f45be05

please copy the NN cluster ID

Solution:1
1. Delete hdata dir (storage dir) and format NN and start the services.

2. Now format NameNode and start the services

Note: In this way, you will lose all your data.

Solution 2:

1. Back to home dir
2. Cd hdata/dfs/data/current
3. Now You needs to change cluster ID of DN to the same as NN.
4. Nano VERSION
5. Change the DN cluster id from old value to new value (in our case new value is: CID-99237010-410d-412e-8ec3-a203bbe991ea).
6. Save and exit
7. Now, come back to home dir and start services (Run start-dfs.sh or start-all.sh)

This problem happens may be your NN is formatted more than once.

For more details, please follow: Installation of Hadoop-2.7-x on Ubuntu

In strong>Hadoop all the logs from starting of service’s till we stop the service’s will be logged in the default “logs” directory (i.e
$HADOOP_HOME/logs by default.) You can change this in the hadoop-env.sh configuration file available in ($HADOOP_HOME/etc/hadoop/ directory).

Now if Data node service is not starting then there will be an java.io.IOException raised which will be logged in $HADOOP_HOME/logs/hadoop-{logged in linux user name will be here}-datanode-ubuntu.log, since it had many details, you can filter for java.io.IOExceptions by “grep {what you are trying to search in the file} filename (i.e grep java.io.IOException $HADOOP_HOME/logs/hadoop-{logged in linux user name will be here}-datanode-ubuntu.log )”, where you will see name node cluster ID and data node cluster ID are different which is the problem. You should me the cluster ID of data node to be the same as name node.

This is how it can be done without loosing any of the data in HDFS.

Navigate to the path where all the temporary file of hadoop will be stored. This path will be set during installation of hadoop int the Edit core-site.xml file located at $HADOOP_HOME/etc/hadoop/.

Once you have navigated to that path navigate to “/dfs/data/current” directory.

Edit the VERSION file, using “nano VERSION” command.

Change the value of clusterID to the clusterID value of data node which was there in the log file of datanode-ubuntu.log (mentioned above)

Save and exit.

Restart the service, once you’r in the $HADOOP_HOME path, run the command sbin/start-dfs.sh

For more details, please follow: Installation of Hadoop-2.7-x on Ubuntu