SpringerLink
Log in

Big Data: Issues, Challenges, and Techniques in Business Intelligence

  • Conference paper
  • First Online:
Big Data Analytics

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 654))

Abstract

During the last decade, the most challenging problem the world envisaged was big data problem. The big data problem means that data is growing at a much faster rate than computational speeds. And it is the result of the fact that storage cost is getting cheaper day by day, so people as well as almost all business or scientific organizations are storing more and more data. Social activities, scientific experiments, biological explorations along with the sensor devices are great big data contributors. Big data is beneficial to the society and business but at the same time, it brings challenges to the scientific communities. The existing traditional tools, machine learning algorithms, and techniques are not capable of handling, managing, and analyzing big data, although various scalable machine learning algorithms, techniques, and tools (e.g., Hadoop and Apache Spark open source platforms) are prevalent. In this paper, we have identified the most pertinent issues and challenges related to big data and point out a comprehensive comparison of various techniques for handling big data problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now
Chapter
USD 29.95
Price excludes VAT (Brazil)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (Brazil)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (Brazil)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Kaisler, S., Armour, F., Espinosa, J.A., Money, W.: Big data: issues and challenges moving forward. In: 2013 46th Hawaii International Conference on System Sciences (HICSS), pp. 995–1004. IEEE (2013)

    Google Scholar 

  2. Katal, A., Wazid, M., Goudar, R.: Big data: issues, challenges, tools and good practices. In: 2013 Sixth International Conference on Contemporary Computing (IC3), pp. 404–409. IEEE (2013)

    Google Scholar 

  3. Fan, J., Han, F., Liu, H.: Challenges of big data analysis. Natl. Sci. Rev. 1(2), 293–314 (2014)

    Article  Google Scholar 

  4. Beyer, M.A., Laney, D.: The Importance of Big Data: A Definition. Gartner, Stamford (2012)

    Google Scholar 

  5. Laney, D.: 3d data management: controlling data volume, velocity and variety. META Group Research Note, 6, 70 (2001)

    Google Scholar 

  6. Minelli, M., Chambers, M., Dhiraj, A.: Big Data, Big Analytics: Emerging Business Intelligence and Analytic Trends for Today’s Businesses. Wiley, New York (2012)

    Google Scholar 

  7. Vossen, G.: Big data as the new enabler in business and other intelligence. Vietnam J. Comput. Sci. 1(1), 3–14 (2014)

    Article  Google Scholar 

  8. Laney, D.: 3D data management: controlling data volume, velocity and variety. META Group Research Note, 6, 70 (2001)

    Google Scholar 

  9. Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Morgan Kaumann Publishers, United States of America (2001)

    MATH  Google Scholar 

  10. Jabin, S., Zareen, F.J.: Biometric signature verification. Int. J. Biom. 7(2), 97–118 (2015)

    Article  Google Scholar 

  11. Jabin, S.: Stock market prediction using feed-forward artificial neural network. Int. J. Comput. Appl. 99(9), 4–8 (2014)

    Google Scholar 

  12. Jabin, S.: Learning classifier systems approach for automated discovery of hierarchical censored production rules. In: Information and Communication Technologies, pp. 68-77. Springer, Berlin (2010)

    Google Scholar 

  13. **ong, H.Y., Alipanahi, B., Lee, L.J., Bretschneider, H., Merico, D., Yuen, R.K., Hua, Y., Gueroussov, S., Najafabadi, H.S., Hughes, T.R., et al.: The human splicing code reveals new insights into the genetic determinants of disease. Science 347(6218), 1254806 (2015)

    Article  Google Scholar 

  14. Kellis, M., Wold, B., Snyder, M.P., Bernstein, B.E., Kundaje, A., Marinov, G.K., Ward, L.D., Birney, E., Crawford, G.E., Dekker, J., et al.: Defining functional DNA elements in the human genome. Proc. Natl. Acad. Sci. 111(17), 6131–6138 (2014)

    Article  Google Scholar 

  15. Tsikerdekis, M., Zeadally, S.: Multiple account identity deception detection in social media using nonverbal behavior. IEEE Trans. Inf. Forensics Secur. 9(8), 1311–1321 (2014)

    Article  Google Scholar 

  16. Schön, D.A., Argyris, C.: Organizational learning: a theory of action perspective. Reis: Revista española de investigaciones sociológicas 77, 345–350 (1997)

    Google Scholar 

  17. Ebner, K., Buhnen, T., Urbach, N.: Think big with big data: identifying suitable big data strategies in corporate environments. In: 2014 47th Hawaii International Conference on System Sciences (HICSS), pp. 3748–3757. IEEE (2014)

    Google Scholar 

  18. MAHOUT (2015). http://www.tutorialspoint.com/mahout/

  19. Che, D., Safran, M., Peng, Z.: From big data to big data mining: challenges, issues, and opportunities. In: Database Systems for Advanced Applications, pp. 1–15. Springer, Berlin (2013)

    Google Scholar 

  20. Hashem, I.A.T., Yaqoob, I., Anuar, N.B., Mokhtar, S., Gani, A., Khan, S.U.: The rise of “big data” on cloud computing: review and open research issues. Inf. Syst. 47, 98–115 (2015)

    Google Scholar 

  21. Spark 0.6.2 (2015). Spark Overview: http://spark.apache.org/docs/0.6.2/

  22. Wu, X., Zhu, X., Wu, G.Q., Ding, W.: Data mining with big data. IEEE Trans. Knowl. Data Eng. 26(1), 97–107 (2014)

    Article  Google Scholar 

  23. Buhl, H.U., Röglinger, M., Moser, D.K.F., Heidemann, J.: Big data. Bus. Inf. Syst. Eng. 5(2), 65–69 (2013)

    Article  Google Scholar 

  24. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  25. Wu, K.: Fastbit: an efficient indexing technology for accelerating data-intensive science. In: Journal of Physics: Conference Series, vol. 16, p. 556. IOP Publishing (2005)

    Google Scholar 

  26. Dittrich, J., Quiane-Ruiz, J.A.: Efficient big data processing in hadoop MapReduce. Proc. VLDB Endowment 5(12), 2014–2015 (2012)

    Article  Google Scholar 

  27. Triguero, I., Peralta, D., Bacardit, J., Garc a, S., Herrera, F.: MRPR: a MapReduce solution for prototype reduction in big data classification. Neurocomputing 150, 331–345 (2015)

    Google Scholar 

  28. Neumeyer, L., Robbins, B., Nair, A., Kesari, A.: S4: distributed stream computing platform. In: 2010 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 170–177 (2010)

    Google Scholar 

  29. Chen, C.P., Zhang, C.Y.: Data-intensive applications, challenges, techniques and technologies: a survey on big data. In: Information Conference on Cloud System and Big Data Engineering, pp. 404–409. IEEE (2013)

    Google Scholar 

  30. Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: the next frontier for innovation, competition, and productivity (2011)

    Google Scholar 

  31. Kim, G.H., Trimi, S., Chung, J.H.: Big-data applications in the government sector. Commun. ACM 57(3), 78–85 (2014)

    Article  Google Scholar 

  32. Stonebraker, M., Hong, J.: Researchers ‘big data crisis; understanding design and functionality. Commun ACM 55(2), 10–11 (2012)

    Article  Google Scholar 

  33. Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, vol. 10, p. 10 (2010)

    Google Scholar 

  34. Driscoll, A.O., Daugelaite, J., Sleator, R.D.: Big data, Hadoop and cloud computing in genomics. J. Biomed. Inf. 46(5), 774–781 (2013)

    Google Scholar 

  35. Simoff, S., Bohlen, M.H., Mazeika, A.: Visual Data Mining: Theory, Techniques and Tools for Visual Analytics, vol. 4404. Springer Science & Business Media (2008)

    Google Scholar 

  36. Sawant, N., Shah, H.: Big data visualization patterns. In: Big Data Application Architecture Q&A, pp. 79–90 (2013)

    Google Scholar 

  37. Apache Software Foundation: The Apache Software Foundation Blog (2014). https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces80

  38. Hadoop: MapReduce Tutorial. http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html#MapReduceTutorial

  39. Apache Storm (2015). https://storm.apache.org/

  40. Gu, L., Li, H.: Memory or time: performance evaluation for iterative operation on hadoop and spark. In: 2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing (HPCC EUC), pp. 721–727. IEEE (2013)

    Google Scholar 

  41. Zheng, X., Zeng, Z., Chen, Z., Yu, Y., Rong, C.: Detecting spammers on social networks. Neurocomputing 159, 27–34 (2015)

    Article  Google Scholar 

  42. Conti, M., Poovendran, R., Secchiero, M.: Fakebook: detecting fake profiles in on-line social networks. In: Proceedings of the 2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012), pp. 1071–1078. IEEE Computer Society (2012)

    Google Scholar 

  43. Anwar, T., Abulaish, M.: Ranking radically influential web forum users. IEEE Trans. Inf. Forensics Secur. 10(6), 1289–1298 (2015)

    Article  Google Scholar 

  44. McKenna, A., Hanna, M., Banks, E., Sivachenko, A., Cibulskis, K., Kernytsky, A., Garimella, K., Altshuler, D., Gabriel, S., Daly, M., et al.: The genome analysis toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. Genome Res. 20(9), 1297–1303 (2010)

    Article  Google Scholar 

  45. Addressing five challenges of Big Data. https://www.progress.com/docs/default-source/default-document-library/Progress/Documents/Papers/Addressing-Five-Emerging-Challenges-of-Big-Data.pdf

  46. Webpopedia: Unstructured Data. http://www.webopedia.com/TERM/U/unstructured_data.html

  47. Marx, V.: Biology: the big challenges of big data. Nature 498(7453), 255–260 (2013)

    Article  Google Scholar 

  48. Assuncao, M.D., Calheiros, R.N., Bianchi, S., Netto, M.A., Buyya, R.: Big data computing and clouds: trends and future directions. J. Parallel Distrib. Comput. 79, 3–15 (2015)

    Article  Google Scholar 

  49. Kambatla, K., Kollias, G., Kumar, V., Grama, A.: Trends in big data analytics. J. Parallel Distrib. Comput. 74(7), 2561–2573 (2014)

    Article  Google Scholar 

  50. Shanahan, J.G., Dai, L.: Large scale distributed data science using apache spark. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2323–2324. ACM (2015)

    Google Scholar 

  51. Stich, V., Jordan, F., Birkmeier, M., Oazgil, K., Reschke, J., Diews, A.: Big data technology for resilient failure management in production systems. In: Advances in Production Management Systems: Innovative Production Management Towards Sustainable Growth, pp. 447–454. Springer, Berlin (2015)

    Google Scholar 

  52. Agrawal, D., Das, S., El Abbadi, A.: Big data and cloud computing: current state and future opportunities. In: Proceedings of the 14th International Conference on Extending Database Technology, pp. 530–533. ACM (2011)

    Google Scholar 

  53. McDaniel, M.A.: Big-brained people are smarter: a meta-analysis of the relation-ship between in vivo brain volume and intelligence. Intelligence 33(4), 337–346 (2005)

    Article  Google Scholar 

  54. Tan, K.H., Zhan, Y., Ji, G., Ye, F., Chang, C.: Harvesting big data to enhance supply chain innovation capabilities: an analytic infrastructure based on deduction graph. Int. J. Prod. Econ. 165, 223–233 (2015)

    Article  Google Scholar 

  55. Chen, M., Mao, S., Liu, Y.: Big data: a survey. Mobile Netw. Appl. 19(2), 171–209 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Jamia Millia Islamia, New Delhi, 110025, India

    Mudasir Ahmad Wani & Suraiya Jabin

Authors
  1. Mudasir Ahmad Wani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suraiya Jabin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mudasir Ahmad Wani .

Editor information

Editors and Affiliations

  1. Jagan Institute of Management Studies, New Delhi, Delhi, India

    V. B. Aggarwal

  2. Department of Computer Science, University of Delhi, New Delhi, Delhi, India

    Vasudha Bhatnagar

  3. Microsoft Innovation Centre, Sri Aurobindo Institute of Technology, Indore, Madhya Pradesh, India

    Durgesh Kumar Mishra

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper

Wani, M.A., Jabin, S. (2018). Big Data: Issues, Challenges, and Techniques in Business Intelligence. In: Aggarwal, V., Bhatnagar, V., Mishra, D. (eds) Big Data Analytics. Advances in Intelligent Systems and Computing, vol 654. Springer, Singapore. https://doi.org/10.1007/978-981-10-6620-7_59

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-6620-7_59

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-6619-1

  • Online ISBN: 978-981-10-6620-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation