Map Join Reduce

Better Essays
Topics: Data
OPTIMIZATION OF MULTISET DATA ANALYSIS ON HADOOP USING MAP JOIN REDUCE

A PROJECT REPORT
Submitted by

SHENBAGA PRIYA.B
09ITR105

SILAMBARASAN.R
09ITR108

VIGNESWARI.A
09ITR125 in partial fulfilment of the requirements for the award of the degree of

BACHELOR OF TECHNOLOGY IN INFORMATION TECHNOLOGY
DEPARTMENT OF INFORMATION TECHNOLOGY SCHOOL OF COMMUNICATION AND COMPUTER SCIENCES

KONGU ENGINEERING COLLEGE
(Autonomous)

PERUNDURAI ERODE – 638 052

APRIL 2013

ABSTRACT

Data analysis is the process of inspecting, cleaning, transforming and modeling data with the goal of highlighting useful information, suggesting conclusions and supporting decision making, which is considerable in cloud computing which allows a large amount of data to be processed over very large clusters. MapReduce is used to handle data in the cloud environment especially in distributed environment because of its excellent scalability and good fault tolerance. But, compared to parallel databases, the efficiency of MapReduce is not efficient when it is adopted to perform complex data analysis which includes joining of multiple data sets in order to compute certain aggregates. A system called Map Join Reduce, which performs complex data analytical task effectively when compared to existing, is proposed. Filtering-join-aggregation model, an extension of MapReduce’s filtering aggregation programming model is introduced. First it performs filtering logic to the data sets and processed in pipelined manner, then groups the output and produces the final result. The significance of our proposal is that, aggregate multiple data sets in one go and thus reduce checkpoints which perform often in existing system and shuffling of intermediate results which results in efficiency of data processing in distributed applications.

INTRODUCTION

In Information Technology, big data is a collection of data sets which is too large and complex that it becomes difficult to process using



References: 1. Afrati.F.N and Ullman.J.D.(2010) ‘Optimizing Joins in a Map-Reduce Environment,’ Proc. 13th Int’l Conf. Extending Database Technology(EDBT ’10). 2. Chuck Lam. (2010) ‘Hadoop in action’, Manning publications. 3. Dawei Jiang, Anthony K. H. Tung, and Gang Chen. (2011) ‘MAP-JOIN-REDUCE: Toward Scalable and Efficient Data Analysis on Large Clusters’, IEEE Transactions on Knowledge and Data Engineering, Vol. 23, No. 9. 4. Dean.J and Ghemawat.S. (2004) ‘MapReduce: Simplified Data Processing on Large Clusters,’ Proc. Operating Systems Design and implementation (OSDI), pp. 137-150. 5. Yang.H.C, Dasdan.A, HsiaoR.L, and Parker.D.S. (2007) ‘Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters,’ Proc. ACM SIGMOD Int’l Conf. Management of Data (SIGMOD ’07).

You May Also Find These Documents Helpful

  • Good Essays

    Map Reduce

    • 320 Words
    • 2 Pages

    Online Test Management System ABSTRACT: This Web Application provides facility to conduct online examination world wide. It saves time as it allows number of students to give the exam at a time and displays the results as the test gets over, so no need to wait for the result. It is automatically generated by the server. Administrator has a privilege to create, modify and delete the test papers and its particular questions. User can register, login and give the test with his specific id, and…

    • 320 Words
    • 2 Pages
    Good Essays
  • Powerful Essays

    transformation, analysis of the transformation technique, Correct decisions relating to selection of the platform and applicable choice of the language for application development. 5.1 General Implementation Discussions Implementation part should perfectly map the design document in a suitable programming language so as to realize the required final and correct product. 5.1.1 Java In this project, for implementation purpose Java is chosen because the programming language. We have used…

    • 1451 Words
    • 6 Pages
    Powerful Essays
  • Satisfactory Essays

    Join

    • 254 Words
    • 2 Pages

    Is this how you expect us to join the website? Well then, I’m just looking for some notes to help me with an internal assessment. As a common teenager, I don’t really expect to have to fill in a 250 word essay and submit it just so I can join a website I will probably only spend 5 minutes on. Hence why this informal essay is on why I shouldn’t do a proper essay for the sake of joining your website. As it turns out, I am studying year 12 classics. I am investigating the differences between the movie…

    • 254 Words
    • 2 Pages
    Satisfactory Essays
  • Better Essays

    Perceptual Maps

    • 1091 Words
    • 5 Pages

    Using Perceptual Maps in Marketing Simulation Janice JohnsonMKT/421 June 5, 2013 Using Perceptual Maps in Marketing Simulation While reading this paper the reader will get a summary of the three major phases from using Perceptual Maps in Marketing Simulation. The phase will be the situation, a recommend solution and the reason why it was chosen, and the results. The relationship between differentiation and positioning will be discussed and whether or not if the repositioning of the product…

    • 1091 Words
    • 5 Pages
    Better Essays
  • Best Essays

    Google Maps vs. Apple Maps

    • 2504 Words
    • 11 Pages

    New Product Success/Failure Paper Apple Maps Vs. Google Maps Intro: Global Positioning Systems (GPS) is a space based satellite navigation system that provides location and time information all over the globe where there is no obstruction to the line of sight to the GPS satellites. The GPS project was developed in 1973 to overcome the limitations of previous navigation systems. It was originally designed for military use by the U.S. Department of Defense. Advances in technology and the…

    • 2504 Words
    • 11 Pages
    Best Essays
  • Powerful Essays

    Pakistan Map

    • 15370 Words
    • 62 Pages

    ! " # %& $ & ' 1 Acknowledgements My thanks go to all the individuals who took time to answer my questions during interviews in the UK and Pakistan and to explain and demonstrate their mapping methodologies and outputs to me. I’m especially grateful to OPPRTI and ASB in Pakistan. This report would not have been possible without their professional and dedicated organisation of my itinerary in Karachi and in Faisalabad and Jaranwala and their openness to all my questions and comprehensive…

    • 15370 Words
    • 62 Pages
    Powerful Essays
  • Good Essays

    The Ghost Map

    • 2841 Words
    • 12 Pages

    Dana Triplett March 1, 2013 Steven Johnson, The Ghost Map. New York, Penguin, 2006. The expansive growth of industrial London awakens an epidemic that seems to kill indiscriminately. Cholera is a disease that had no discernible cause, much less a cure, during the nineteenth century. People are dying regardless of their social class or living conditions. Looking for a method to the madness that is cholera, Doctor John Snow begins a quest to investigate the spread of the disease throughout a neighborhood…

    • 2841 Words
    • 12 Pages
    Good Essays
  • Better Essays

    Karnaugh Map

    • 1639 Words
    • 7 Pages

    Karnaugh map From Wikipedia, the free encyclopedia Jump to: navigation, search | This article includes a list of references, but its sources remain unclear because it has insufficient inline citations. Please help to improve this article by introducing more precise citations where appropriate. (June 2010) | For former radio station KMAP (1962-1968) in Dallas-Fort Worth, see KRLD-FM. An example Karnaugh map The Karnaugh map (K-map for short), Maurice Karnaugh's 1953 refinement of Edward…

    • 1639 Words
    • 7 Pages
    Better Essays
  • Good Essays

    Topographic Map

    • 395 Words
    • 2 Pages

    MS 217 Dennis Borzakov Class 723 January 15, 2013 Problem HOW IS A TOPOGRAPHIC MAP MADE Hypothesis I think that to make a topographic map you have to see the form of the object from up top. To do this you need a satellite image. These images are called aerial photographs. Using elevation calculators and ground measures cartographers then make topographic maps. Materials • Clay model landform • Water tinted with food coloring • Transparency • Clear…

    • 395 Words
    • 2 Pages
    Good Essays
  • Satisfactory Essays

    Map Analysis

    • 768 Words
    • 4 Pages

    Study guide—Final Exam (April 26, 2007: 3:00 pm) GIS 3015 (Map Analysis) Spring 2007 OVERARCHING THEMES (5-10 questions at the most) --Understand that maps are human creations and imperfect though useful representations of the land surface, understand why we use (though not the specifics of each one) grid systems, different projections. Understand that there of many types, and a few specifics: political, physical, cadastral, chloropleth, why we generalize, basics of topographic lines COMPUTER…

    • 768 Words
    • 4 Pages
    Satisfactory Essays