Can we recover a hadoop cluster where all data nodes are down

[DataFlair Team]

Can we recover a hadoop cluster where all data nodes are down ?
I found checkpoint node and backup node keeps the edit logs and fs image in another system.
So using checkpoint can we restore all data nodes data

[DataFlair Team]

Each DataNode sends a Heartbeat message to the NameNode periodically.
A network partition can cause a subset or all of the DataNodes to lose connectivity with the NameNode.
The NameNode detects this condition by the absence of a Heartbeat message.
The NameNode marks DataNodes without recent Heartbeats as dead and does not forward any new IO requests to them.
Any data that was registered to a dead DataNode is not available to HDFS any more.
DataNode death may cause the replication factor of some blocks to fall below their specified value.
The NameNode constantly tracks which blocks need to be replicated and initiates replication whenever necessary.

So we can recover any data loss occured.

FsImage->An fsimage file contains the complete state of the file system at any particular point of time.
Every file system modification is assigned a unique transaction ID.
An fsimage file represents the file system state after all modifications up to a specific transaction ID.

Edit Logs:-Log files that lists each file system change (file creation, deletion or modification)
that was made after the most recent fsimage.

To see edit logs and fsimage file location open the hdfs-site.xml file.

[Author]

Posts