Reduced Time Compression in Big Data Using MapReduce Approach and Hadoop

Abstract

An exponential rise has been observed in the data volume over the time when considering a real time environment. A phenomenal feature termed as ‘Predictability’ helps in predicting and portraying related data to the user according to their needs. Moreover, classification of Big Data is usually a tedious and lengthy task. The technique of MapReduce Framework performs the data processing that being paralleled by data distribution in small chunks through the clusters. This Map Reduce technique is being proposed which is employed to process heterogeneous data items. Few issues that are being targeted in the existing paper include associating climatologically and meteorological information with large variety of farming decisions. Using the well-known MapReduce framework the above issues and challenges can be resolved. The existing paper proposes empirical techniques of climate classification and prediction by adopting Co-EANFS (Co-Effective and Adaptive Neuro-Fuzzy System) approach for data handling. Furthermore, the paper examines association rule mining too, which is being implemented for examining the best crop production by relying upon the soil and weather condition. Lastly, a technique is proposed for managing various levels such as preprocessing, clustering, classification and prediction. First, the weather dataset is being collected which undergoes processing; thereafter the proposed model is implemented which results in formation of cluster data sets linked to each season. For evaluating the performance, accuracy predictions generated by Co-EANFS is used which being formulated with varying no: of inputs and variables. The proposed framework acquires least execution time.