Breaking News
Loading...

Writing and reading file in HDFS





ÄWriting a file into HDFS


F 
Say the client wants to write a file into HDFS ,it asks namenode where it can write these files.The namenode checks its metadata and finds the datanodes where it can write this file.This file is divided into blocks if its size is more than the default block size.These blocks are copied into various datanodes considering the replication factor.
 
Client request Name node to provide which data nodes to write

FClient writes block to one data node directly

FData node then forwards write request to next data node(replication)

FCycle repeats for next block 

Reading a file in HDFS: 
Ä
 

ÄClient receives list of data nodes for each block. Name node provides rack local nodes first

ÄPicks the first data node for each block

ÄReads blocks sequentially
Ä
Ä
 
- See more at: http://labstrikes.blogspot.in/2012/08/adsense-middle-blog-post.html#sthash.gQgSkqx8.dpuf
 
Toggle Footer