Say File1.txt is the file which needs to be stored in HDFS.This file is splited up to blocks A, C of size 64 MB(default size,can be modified) .These blocks are stored in different datanodes and the information about these blocks i.e the metadata about these blocks is stored in namenode.
When a new block comes in ,how does the namenode know which datanode to place that block?.
For this ,there is a communication which happens between Namenode and datanode.
- Data node sends heartbeats every 3 seconds via a TCP handshake
- ÄEvery 10thheart beat is a block report
- ÄNamenode builds metadata from block report
- ÄCommunication happens through the port defined for Namenode