Big Data Architecture, Goals and Challenges
Big Data Architecture, Goals and Challenges
Cipson Jose Chiriyankandath
Dakota State University
Abstract
Big Data inspired data analysis is matured from proof of concept projects to an influential tool for decision makers to make informed decisions. More and more organizations are utilizing their internally and externally available data with more complex analysis techniques to derive meaningful insights. This paper addresses some of the architectural goals and challenges for Big Data architecture in a typical organization.
Overview
In this fast paced information age, there are many different sources on corporate networks and internet is collecting massive amounts of data, but there is a significant difference in this data compared to the conventional data, much of this data is semi-structured or unstructured and not residing in conventional databases. “Big data” is essentially a huge data set that scales to multiple petabytes of capacity; it can be created, collected, collaborated, and stored in real-time or any other way. However, the challenge with big data is that it is not easily handled using traditional database management tools. It typically consists of unstructured data, which includes text, audio and video files, photographs and other data (Kovar, 2012). The aim of this paper is to examine the concepts associated with the big data architecture, as well as how to handle, process, and effectively utilize big data internally and externally to obtain meaningful and actionable insights.
How Big Data is Different?
Big data is the latest buzzword in the tech industry, but what exactly makes it different from traditional BI or data analysis? According to MIT Sloan Management Review, big data is described as “data that is either too voluminous or too unstructured to be managed and analyzed through traditional means” (Davenport, Thomas, Barth, & Bean, 2012). Big data is unlike conventional mathematical intelligence, where a simple sum of
Cipson Jose Chiriyankandath
Dakota State University
Abstract
Big Data inspired data analysis is matured from proof of concept projects to an influential tool for decision makers to make informed decisions. More and more organizations are utilizing their internally and externally available data with more complex analysis techniques to derive meaningful insights. This paper addresses some of the architectural goals and challenges for Big Data architecture in a typical organization.
Overview
In this fast paced information age, there are many different sources on corporate networks and internet is collecting massive amounts of data, but there is a significant difference in this data compared to the conventional data, much of this data is semi-structured or unstructured and not residing in conventional databases. “Big data” is essentially a huge data set that scales to multiple petabytes of capacity; it can be created, collected, collaborated, and stored in real-time or any other way. However, the challenge with big data is that it is not easily handled using traditional database management tools. It typically consists of unstructured data, which includes text, audio and video files, photographs and other data (Kovar, 2012). The aim of this paper is to examine the concepts associated with the big data architecture, as well as how to handle, process, and effectively utilize big data internally and externally to obtain meaningful and actionable insights.
How Big Data is Different?
Big data is the latest buzzword in the tech industry, but what exactly makes it different from traditional BI or data analysis? According to MIT Sloan Management Review, big data is described as “data that is either too voluminous or too unstructured to be managed and analyzed through traditional means” (Davenport, Thomas, Barth, & Bean, 2012). Big data is unlike conventional mathematical intelligence, where a simple sum of
References: Anuganti, V. (2012). Typical “big” data architecture. Venu Anuganti Blog. Retrieved from http://venublog.com/2012/11/30/typical-big-data-architecture/ Davenport, T. H., Barth, P., & Bean, R. (2012). How 'Big Data ' is different. MIT Sloan Management Review 54(1), 22-24. Kovar, J. F. (2012). Big data is the next big thing. CRN, (1320), 12-n/a. Oracle Raden, N. (2012). Big data analytics architecture: Putting all your eggs in three baskets. Hired Brains, Inc