bigdata
4.
Big Data
4.1
Introduction
In 2004, Wal-Mart claimed to have the largest data warehouse with 500 terabytes storage
(equivalent to 50 printed collections of the US Library of Congress). In 2009, eBay storage amounted to eight petabytes (think of 104 years of HD-TV video). Two years later, the Yahoo warehouse totalled 170 petabytes1 (8.5 times of all hard disk drives created in 1995)2. Since the rise of digitisation, enterprises from various verticals have amassed burgeoning amounts of digital data, capturing trillions of bytes of information about their customers, suppliers and operations. Data volume is also growing exponentially due to the explosion of machine-generated data (data records, web-log files, sensor data) and from growing human engagement within the social networks.
The growth of data will never stop. According to the 2011 IDC Digital Universe Study, 130 exabytes of data were created and stored in 2005. The amount grew to 1,227 exabytes in 2010 and is projected to grow at 45.2% to 7,910 exabytes in 2015.3 The growth of data constitutes the “Big
Data” phenomenon – a technological phenomenon brought about by the rapid rate of data growth and parallel advancements in technology that have given rise to an ecosystem of software and hardware products that are enabling users to analyse this data to produce new and more granular levels of insight.
Figure 1: A decade of Digital Universe Growth: Storage in Exabytes
1
2
3
Error! Reference source not found.3
Ovum. What is Big Data: The End Game. [Online] Available from: http://ovum.com/research/what-is-big-data-theend-game/ [Accessed 9th July 2012].
IBM. Data growth and standards. [Online] Available from: http://www.ibm.com/developerworks/xml/library/xdatagrowth/index.html?ca=drs- [Accessed 9th July 2012].
IDC. The 2011 Digital Universe Study: Extracting Value from Chaos. [Online] Available from:
Big Data
4.1
Introduction
In 2004, Wal-Mart claimed to have the largest data warehouse with 500 terabytes storage
(equivalent to 50 printed collections of the US Library of Congress). In 2009, eBay storage amounted to eight petabytes (think of 104 years of HD-TV video). Two years later, the Yahoo warehouse totalled 170 petabytes1 (8.5 times of all hard disk drives created in 1995)2. Since the rise of digitisation, enterprises from various verticals have amassed burgeoning amounts of digital data, capturing trillions of bytes of information about their customers, suppliers and operations. Data volume is also growing exponentially due to the explosion of machine-generated data (data records, web-log files, sensor data) and from growing human engagement within the social networks.
The growth of data will never stop. According to the 2011 IDC Digital Universe Study, 130 exabytes of data were created and stored in 2005. The amount grew to 1,227 exabytes in 2010 and is projected to grow at 45.2% to 7,910 exabytes in 2015.3 The growth of data constitutes the “Big
Data” phenomenon – a technological phenomenon brought about by the rapid rate of data growth and parallel advancements in technology that have given rise to an ecosystem of software and hardware products that are enabling users to analyse this data to produce new and more granular levels of insight.
Figure 1: A decade of Digital Universe Growth: Storage in Exabytes
1
2
3
Error! Reference source not found.3
Ovum. What is Big Data: The End Game. [Online] Available from: http://ovum.com/research/what-is-big-data-theend-game/ [Accessed 9th July 2012].
IBM. Data growth and standards. [Online] Available from: http://www.ibm.com/developerworks/xml/library/xdatagrowth/index.html?ca=drs- [Accessed 9th July 2012].
IDC. The 2011 Digital Universe Study: Extracting Value from Chaos. [Online] Available from: