Posts

INTRODUCTION FOR MAP REDUCE

Hadoop had two main indispensable components HDFS and Map Reduce. About Hadoop eco system and HDFS, we already discussed in another blog with detailed explanation if you are not gone through, check out those blogs (hdfsblog-coming soon, hadoopecosystem-coming soon). It will  be helpful to understand more about Hadoop Technology.     HDFS is the distributed file system for Hadoop and map reduce is distributed processing engine where can we do cleansing, transformation etc…. TechnicalTerms You need to Know Before stepping into Big Data Processing Map Reduce does :   It is basically a distributed processing engine with batch processing technique. Mostly used for data processing. Programming language map reduce supports: Java, Python, C++, Ruby, Perl are some major languages supported by map reduce. by default, java. java will give better performance than any other language in map reduce. Advantages of map reduce:     it is parallel processing ...
Post a Comment
Read more

Technical Terms You need to Know Before stepping into Big Data Processing

  List Of Technical Terms  Cluster :  group of computers or servers. Parallel processing :  group of computers or servers used to do same task.   Like Multiple developers hired to develop same application by splitting their tasks. Similar to that if we have 320 MB data to process and have 5 machines, Hadoop will split it into five 64 MB chunks then five parallelly processes by 5 machines. It   will reduce 4 times processing time. Concurrency :  Single thread or process will handle multiple works or tasks in a time period. Example : Technically – a process got task to retrieve 100 accounts data through API. API will accept only one account per request, process should make 100 API call to complete that task.   Normally, a process will call API for account1 and wait ideally until get response. In concurrent scenario, meanwhile it waiting for Api response for Account1, it will send requests for other accounts also to API. Suppose, process t...
Post a Comment
Read more