top-rated free essay

Big Data Issues Challenges

By faeezanoor Apr 18, 2015 1673 Words



Every organizations are now speaking about Big Data and some has made it into practice. The initiative of harnessing data to amplify capabilities in achieving organizational objectives has made it possible for practitioner as well as senior management in their decision making.

In Malaysia, the culture of acceptance of Big Data to help in decision making is agreeable. However, the infrastructure for adoption are not fully understood. Nevertheless, the public sector and private sector are in the transition phase from curiosity and enthusiasm to buy in and across startup.


This report illustrate the issues and challenges in more detail of Big Data in Malaysia respectively.


Every organization has the opportunity to leverage big data to its advantage – to drive accurate and timely decisions that can materially affect its business and organizational goals. Following are the challenges in implementing Big Data:


Some of the organizations viewed that their data assets has effectively deriving tangible benefits. From a discussion with Greg Whalen, Pivotal’s Chief Data Scientist, Asia Pacific and Japan, revealed that many organizations are using their existing data to identify new correlations which provide profits through their manipulation of data history. Some of the managers believe they are already using their data assets effectively with the newer approaches in data mining, exploratory visualization and comparison across external data sets may shed new light on unknown opportunities. Studies shown that organizations has increasingly spend 25 percent on big data in 2014 compared to 2013 as a turning point investment. Meanwhile, the ICT industries has an increasing technical fragmentation in big data, therefore, there are opportunities for boutique firms in Malaysia to meet specialist technical needs for global market.


Due in improving customer service and experience, retention, acquisition, cross-sell and up-sell to be competitive in the industry, big data was held important for businesses. In fact, the end use for big data are used to monitor the high ranking of social trends of social data. Therefore, the adoption of big data initiative projects readiness in Malaysia are high amongst managers rather than the personnel. Studies shown that the interest of big data is primarily driven by improving business growth and profit instead of efficiency. This also means that the managers need to align the external and internal goals of having big data. For example, the approach in the change of product and service development would need data to increase operational efficiency. Managers might value to think of risk management, customer retention, customer acquisition, product and service innovation, customer behavioral profiling and forecasting supply and demand. On the other hand, the the practitioner had more focus on internal operation, such as supply chain monitoring and infrastructure and asset monitoring. Therefore, the business need to understand the needs of applying big data in order to reduce cost while increasing the value.


The vary of organizations helps to produce sustainable Big Data ecosystem in desired skills acquisition between ICT and non-ICT. Generally, ICT firms are participating in big data ecosystem as solution providers while the non-ICT firms are developing deep analytical capabilities pertinent to their business needs. Some organizations may participate in the Big Data ecosystem as both producers and consumers, especially organizations in the ICT and marketing services sectors. However, due to the hype surrounding techniques there is a need to also promote the value of fundamental applied math and statistics skills. The most commonly desired skills are special data analysis, modeling and simulation. The intermediate and advanced are the following skills: a) Big and Distributed Data (eg. Hadoop, MapReduce)

b) Algorithms (eg. computational complexity, CS theory)
c) Machine Learning (eg. decision trees, neural nets, SVM, clustering) d) Back-End Programming (eg. JAVA/Rails/Objective C)
e) Visualization (eg. statistical graphics, mapping, web-based dataviz)


Capabilities are desired characteristics and features that may encountered by many industry segments, be applicable to multiple end-uses and/or rely on some combination of skills to produce and consume. In order to help the organization to perform analyses and make decisions much more rapidly, they need to go through the sheer volumes of data. Capabilities are beyond having the access to the data - it is the handling of the data that creates value. The top three priority is to have a real time insights from realtime data streams, uncovering patterns (e.g. segments, correlations) from multi-structured datasets, and data visualization to reveal patterns and trends. To do so, it takes lot of understanding to visualize the data as part of your analysis in a right shape. For example, if the data comes from social media content, you need to know who the user is in a general sense - such as a customer using a particular set of products - and understand what it is you’re trying to visualize out of the data. Without some sort of context, visualization tools are likely to be of less value to the user. One solution to this challenge is to have the proper domain expertise in place. The organization need to make sure the people who analyze the data have a deep understanding of where the data comes from, what audience will be consuming the data and how that audience will interpret the information.


Data is the new oil. The wider social and business impact of Big Data initiatives is dependent on the quality of data and information on which they are based. The value of data for decision making exist only if the data is accurate and timely. It becomes even more pronounced when considering the volumes of information involved. The organization need to have a data governance or data management in place to ensure the data is clean. The data source would help most organization in getting started by combining the internal and external data source. Data environments whether open or proprietary are becoming increasingly complex as new business models emerge. In Malaysia, complex social problems such as traffic congestion or local crime could be addressed by open data and crowd-sourcing approaches, in both diagnosing problems and in providing feedback mechanisms from citizens. Open data from government sources support the big data ecosystem especially for social problems that have many dimension. For example, access to health care is also a function of transportation, location and income. Ministries can utilize internal data management for service optimization. However, the weariness of the bureaucracy and red tape would be a hindrance to to the big data initiatives. Moreover, the issues of privacy, resource cost and perception of Malaysian professionals to have public cloud. There also a concern on organizations which only have a single source of data rather than combining the internal and external sources. External sources may benefit some organization in getting started with the basic goal of identifying data sources. For example, respondent who were likely to highly-value their customer behavioral profiling will depend on third party data such as social media. In addition, the public cloud vendors need to identify and address the concern of privacy, resource sot, and perceptions specific to Malaysian professionals.


1. To foster a data data-centric organizational culture, where organizational processes, decisions, and objectives are supported by leveraging a variety of data resources. 2. To equip the organization to manage and make decisions based on real-time data, in order to remain competitive. 3. To cultivate multi-disciplinary data science teams, consisting of specialists with advanced skills in mathematics and statistics, domain experts, and others. 4. To combine data from internal sources within organization together with external sources (including public and proprietary third-party sources) to spur innovation. 5. To create a ‘data dictionary’ of data assets relevant to your organization, to minimize friction for data science projects across the organization. 6. Large organizations, and especially government, should make concerted efforts to release data to the public in machine consumable form to unlock value hidden within their data assets.


This report shows the progress level of maturity of Big Data analytics in Malaysia. Even though the support was there at the executive levels, however, the aspirations and enthusiasm for Big Data is not uniformly matched with resource commitment. There are several inhibitors that prevent organization in making headway with Big Data projects, including recruiting the appropriate skills, the ideation of novel, high value use cases and access to datasets. Big Data involves not just technological evolution, but a cultural shift in the way of work and share knowledge. These values include an increasing emphasis on transparency and continuous innovation. The advent in Big Data brings rapid change but also emphasizes interest in classical disciplines such as mathematics and statistics. The pitfalls in the use of Big Data should be at the forefront in considering adoption. The privacy rights of individuals will also constrain how organizations collect, manage and distribute data. However, we believe the productive value of Big Data analytics in societies outweighs drawbacks if these concerns are respected and managed carefully. We have no doubt that the Big Data analytics industry will continue to mature and evolve in unexpected ways in Malaysia that will support the nation’s competitiveness in the global economy.


1. Big Data In Malaysia : Emerging Sector Profile 2014. (2014) (1st ed.). Malaysia. Retrieved from 2. Addressing Data Volume, Velocity, and Variety with IBM InfoSphere Streams V3.0. (2014) (1st ed.). 3.,. (2014). What is Big Data?. Retrieved 5 December 2014, from

Cite This Document

Related Documents


    ...without the superfluous and simply talk of evaluation (period!)? Critics might object that terms, expressions, definitions are simply social conventions designed to cover meanings that are, in turn, social conventions, and that therefore one word is as good as another. That sounds correct. But is it reasonable? Here is a perfectly absurd but...

    Read More
  • The Big Data Challenges

    ...Click the link above to submit your assignment. Students, please view the "Submit a Clickable Rubric Assignment" in the Student Center. Instructors, training on how to grade is within the Instructor Center. Case Study 1: The Big Data Challenges Due Week 3 and worth 100 points Big Data is a term used to describe the voluminous amount of st...

    Read More
  • Big Data Challenges

    ...The Big Data Challenges Lamin A Tunkara CIS 500 – Information Systems for Decision Making Dr. Edwin OTTO July 16, 2013 Abstract The paper explores the challenges of big data and the strategies deployed by Volvo Car Corporation in instituting an information technology infrastructural tool that helped the company ...

    Read More
  • Volvo’s Big Data Challenges

    ...Information Systems for Decisions CIS 500 Volvo’s Big Data Challenges In the ever evolving world of technology, there are many leaps forward that many companies can choose to leverage in an attempt to bring a better product to the market. Over the last few years, cloud computing has evolved to the point where it has become a very viab...

    Read More
  • Big Data

    ...Big Data Management: Possibilities and Challenges The term big data describes the volumes of data generated by an enterprise, including Web-browsing trails, point-of-sale data, ATM records, and other customer information generated within an organization (Levine, 2013). These data sets can be so large and complex that they become difficult to ...

    Read More
  • The four V's of Big Data

    ...The four V’s of volume, velocity, variety and veracity are commonly used to characterise different aspects of big data. Discuss. Big data refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. We assume that, as technology advances over time, the size of dataset...

    Read More
  • Big Data

    ...Banks, Big Data and High-Performance Analytics Insights on Turning Big Data into Competitive Advantage A selection of articles that appeared in Big Data = Big Gains, a special digital publication by Bank Systems & Technology Table of Contents Turning Big Data into Competitive Advantage . . . . . . . . . . . . . . . . . ...

    Read More
  • Big Data

    ...sales data in real time. “With the SAP HANA solution, we needed a partner that was ready to support us in every way possible. By helping us to stay in budget and to adopt a system that could grow in line with company requirements, Huawei really delivered.” Federico Vecchiatti, Head of IT, Unicomm “Before they started the SAP HANA project,...

    Read More

Discover the Best Free Essays on StudyMode

Conquer writer's block once and for all.

High Quality Essays

Our library contains thousands of carefully selected free research papers and essays.

Popular Topics

No matter the topic you're researching, chances are we have it covered.