Abstract
The smart grid surpasses the traditional grid in terms of the type, scale, and speed of data generated during the transition. In addition to monitoring grid operations, the smart grid also focuses on gathering power consumption data from various user appliances. This necessitates the implementation of big data technology to efficiently manage, analyze, and even schedule grid operations. By doing so, the smart grid can operate with enhanced precision and efficiency while swiftly responding to user demands.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Oussous A et al (2017) Big data technologies: a survey. J King Saud Univ Comput Inf Sci https://doi.org/10.1016/j.jksuci.2017.06.001
Chunming T, ** H, Zhikang S et al (2017) Big data issues in smart grid—a review. Renew Sustain Energ Rev 79:1099–1107. https://doi.org/10.1016/j.rser.2017.05.134
Farhangi H (2010) The path of the smart grid. Power Energy Mag IEEE 8(1):18–28
DOE., Advancement of synchrophasor technology in ARRA Projects. https://www.smartgrid.gov/recovery_act/program_publications.html
DOE., Summary of the North American synchrophasor initiative (NASPI) activity area. https://www.naspi.org/documents
Yuan J, Shen J, Pan L et al (2014) Smart grids in China. Renew Sustain Energy Rev 37(3):896–906
Klump R, Agarwal P, Tate JE et al (2010) Lossless compression of synchronized phasor measurements. In: Proceedings of the IEEE power and energy society general meeting. Minneapolis, MN, USA, pp 1–7
Depuru SSSR, Wang L, Devabhaktuni V (2011) Smart meters for power grid: challenges, issues, advantages and status. Renew Sustain Energy Rev 15(6):2736–2742
Kabalci Y (2016) A survey on smart metering and smart grid communication. Renew Sustain Energy Rev 57:302–318
Alahakoon D, Yu X (2016) Smart electricity meter data intelligence for future energy systems: a survey. IEEE Trans Ind Inf 12(1):425–436
Shuai Z, Sun Y, Shen Z et al (2016) Microgrid stability: classification and a review. Renew Sustain Energy Rev 58:167–179
Electric Power Group, Wind farm oscillation detection and mitigation. https://www.smartgrid.gov/recovery_act/project_information.htm
Short TA (2013) Advanced metering for phase identification, transformer identification, and secondary modeling. IEEE Trans Smart Grid 4(2):651–658
Overholt P, Kosterev D, Eto J et al (2014) Improving reliability through better models: using synchrophasor data to validate power plant models. IEEE Power Energy Mag 12(3):44–51
Luan W, Peng J, Maras M et al (2015) Smart meter data analytics for distribution network connectivity verification. IEEE Trans Smart Grid 6(4):1–1
Hajnoroozi AA, Aminifar F, Ayoubzadeh H (2015) Generating unit model validation and calibration through synchrophasor measurements. IEEE Trans Smart Grid 6(1):441–449
Erlinghagen S, Lichtensteiger B, Markard J (2015) Smart meter communication standards in Europe—a comparison. Renew Sustain Energy Rev 43:1249–1262
Zhou K, Yang S (2016) Understanding household energy consumption behavior: the contribution of energy big data analytics. Renew Sustain Energy Rev 56:810–819
Aminifar F, Fotuhi-Firuzabad M, Shahidehpour et al (2011) A probabilistic multistage PMU placement in electric power systems. IEEE Trans Power Deliv 26(2):841–849
Gopakumar P, Chandra G, Reddy M et al (2013) Optimal placement of PMUs for the smart grid implementation in Indian power grid—a case study. Front Energy 7(3):358–372
Xu B, Abur A (2004) Observability analysis and measurement placement for systems with PMUs. In: Proceedings of the IEEE PES power systems conference and exposition, vol 2, no 10, pp 943–946
Huang J, Wu N, Ruschmann M (2014) Data-availability-constrained placement of PMUs and communication links in a power system. IEEE Syst J 8(2):483–492
Li Q, Cui T, Weng Y, Negi R et al (2013) An information-theoretic approach to PMU placement in electric power systems. IEEE Trans Smart Grid 4(1):446–456
Sodhi R, Srivastava SC, Singh SN (2011) Multi-criteria decision-making approach for multistage optimal placement of phasor measurement units. IET Gener Transm Distrib 5(2):181–190
Domingos P, Hulten G (2000) Mining high-speed data streams. In: Proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining, pp 71–80
Basu C, Agrawal A, Hazra J et al (2014) Understanding events for wide-area situational awareness. In: Proceedings of the innovative smart grid technologies conference. IEEE, pp 1–5
Peppanen J, Reno MJ, Thakkar M et al (2015) Leveraging AMI data for distribution system model calibration and situational awareness. IEEE Trans Smart Grid 6(4):2050–2059
Gol M, Abur A (2015) A fast decoupled state estimator for systems measured by PMUs. IEEE Trans Power Syst 30(5):2766–2771
Zhao J, Zhang G, Das K et al (2015) Power system real-time monitoring by using PMU-based robust state estimation method. IEEE Trans Smart Grid 7(1):1
**e L, Chen Y, Kumar PR (2014) Dimensionality reduction of synchrophasor data for early event detection: linearized analysis. IEEE Trans Power Syst 29(6):2784–2794
Dahal OP, Brahma SM, Cao H (2014) Comprehensive clustering of disturbance events recorded by phasor measurement units. IEEE Trans Power Deliv 29(3):1390–1397
Khan AR, Mahmood A, Safdar A et al (2016) Load forecasting, dynamic pricing and DSM in smart grid: a review. Renew Sustain Energy Rev 54:1311–1322
Selakov A, Cvijetinović D, Milović L et al (2014) Hybrid PSO–SVM method for short-term load forecasting during periods with significant temperature variations in city of Burbank. Appl Soft Comput 16(3):80–88
Deihimi A, Orang O, Showkati H (2013) Short-term electric load and temperature forecasting using wavelet echo state networks with neural reconstruction. Energy 57(3):382–401
Zhu L, Wu QH, Li MS et al (2013) Support vector regression-based short-term wind power prediction with false neighbors filtered. In: Proceedings of the international conference on renewable energy research and applications. IEEE, pp 740–744
Rocio Cogollo M, Velasquez JD (2014) Methodological advances in artificial neural networks for time series forecasting. IEEE Lat Am Trans 12(4):764–771
Huang SC, Lu CN, Lo YL (2015) Evaluation of AMI and SCADA data synergy for distribution feeder modeling. IEEE Trans Smart Grid 6(4):1–1
Berrisford AJ (2013) A tale of two transformers: an algorithm for estimating distribution secondary electric parameters using smart meter data, pp 1–6
Jokar P, Arianpoo N, Leung VCM (2016) Electricity theft detection in AMI using customers’ consumption patterns. IEEE Trans Smart Grid 7(1):216–26
Kwac J, Rajagopal R (2016) Data-driven targeting of customers for demand response. IEEE Trans Smart Grid 7(5):2199–2207
Peppanen J, Reno MJ, Broderick RJ et al (2016) Distribution system model calibration with big data from AMI and PV inverters. IEEE Trans Smart Grid 7(5):2497–2506
Pan S, Morris T, Adhikari U (2015) Develo** a hybrid intrusion detection system using data mining for power systems. IEEE Trans Smart Grid 6(6):134–143
Bayrak G, Kabalci E (2016) Implementation of a new remote islanding detection method for wind–solar hybrid power plants. Renew Sustain Energy Rev 2016(58):1–15
Li C, Higuma K, Watanabe M et al (2008) Monitoring and estimation of interarea power oscillation mode based on application of CampusWAMS. Turk J Agric Food Sci Technol 3(1)
Dasgupta S, Paramasivam M, Vaidya U et al (2015) PMU-based model-free approach for real-time rotor angle monitoring. IEEE Trans Power Syst 30(5):2818–2819
Liu Y, Zhan L, Zhang Y et al (2015) Wide-area measurement system development at the distribution level: an FNET/GridEye example. IEEE Trans Power Deliv 31(2):1–1
Chai J, Liu Y, Guo J et al (2016) Wide-area measurement data analytics using FNET/GridEye: a review. In: Proceedings of the power systems computation conference, pp 1–6
Zhou D, Guo J, Zhang Y et al (2016) Distributed data analytics platform for wide-area synchrophasor measurement systems. IEEE Trans Smart Grid 7(5):2397–2405
Kamwa J, Beland G, Trudel et al (2006) Wide-area monitoring and control at hydroquébec: past, present and future. In: Proceedings of the power engineering society general meeting
e-terraphasorpoint 3.0. http://www.alstom.com/Global/Grid/Resources/Documents/Automation/NMS/e-terraphasorpoint%203.0.pdf
Siemens (2015) http://www.engerati.com/resources/efficient-network-integrationrenewable-energy-resources-distribution-level
NARI (2015) http://www.naritech.cn/html/jie166.shtm
Singh M, Ali A (2016) Big data analytics with Microsoft HDInsight in 24h. Sams Teach Yourself: Big Data, Hadoop, and Microsoft Azure for better business intelligence
Rusitschka S, Eger K, Gerdes C (2010) Smart grid data cloud: a model for utilizing cloud computing in the smart grid domain. In: Proceedings of the IEEE international conference on smart grid communications, pp, 483–488
Birman KP, Ganesh L, Renesse RV et al (2012) Running smart grid control software on cloud computing architectures. In: Proceedings of the workshop on computational needs for the next generation electric grid
Baek J, Vu QH, Liu JK et al (2015) A secure cloud computing based framework for big data information management of smart grid. IEEE Trans Cloud Comput. 3(2):233–244
Simmhan Y, Prasanna V, Aman S et al (2013) Cloud-based software platform for big data analytics in smart grids. Comput Sci Eng 15(4):38–47
Khan M (2015) Hadoop performance modeling and job optimization for big data analytics. Brunel University, London
Markovic DS, Zivkovic D, Branovic I et al (2013) Smart power grid and cloud computing. Renew Sustain Energy Rev 24(1):566–577
Fang B, Yin X, Tan Y et al (2016) The contributions of cloud technologies to smart grid. Renew Sustain Energy Rev 59:1326–1331
Apache Hadoop. http://hadoop.apache.org/
Zhang Y, Chen S, Wang Q et al (2015) I2MapReduce: incremental MapReduce for mining evolving big data. IEEE Trans Knowl Data Eng 27(7):1
**ng EP, Ho Q, Dai W et al (2015) Petuum: a new platform for distributed machine learning on big data. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 1–1
Shyam R, Bharathi GHB, Sachin KS et al (2015) Apache spark a big data analytics platform for smart grid. 21:171–178
Apache Spark. http://spark.apache.org/
Li D (2014) Machine learning aided decision making and adaptive stochastic control in a hierarchical interactive smart grid. Dissertations and theses—grad-works
Wu X, Zhu X, Wu GQ et al (2014) Data mining with big data. IEEE Trans Knowl Data Eng 26(1):97–107
Al-Jarrah OY, Yoo PD, Muhaidat S et al (2015) Efficient machine learning for big data: a review. Big Data Res 2(3):87–93
Khazaei J, Fan L, Jiang W et al (2016) Distributed prony analysis for real-world PMU data. Electr Power Syst Res 133:113–120
Shaker H, Zareipour H, Wood D (2016) A data-driven approach for estimating the power generation of invisible solar sites. IEEE Trans Smart Grid 7(5):2466–2476
Shaker H, Zareipour H, Wood D (2016) Estimating power generation of invisible solar sites using publicly available data. IEEE Trans Smart Grid 7(5):2456–2465
Taieb SB, Huser R, Hyndman RJ et al (2016) Forecasting uncertainty in electricity smart meter data by boosting additive quantile regression. IEEE Trans Smart Grid 7(5):2448–2455
Partridge M, Calvo RA (1998) Fast dimensionality reduction and simple PCA. Intell Data Anal 2(1–4):203–214
Poekaew P, Champrasert P (2015) Adaptive-PCA: an event-based data aggregation using principal component analysis for WSNs. In: Proceedings of the international conference on smart sensors and application. IEEE
Nuchprayoon S (2014) Electricity load classification using K-means clustering algorithm, Proceedings of the Brunei international conference on engineering and technology. IET, pp 1–5
Yang H, Zhang J, Qiu J et al (2016) A practical pricing approach to smart grid demand response based on load classification. IEEE Trans Smart Grid 1–1
Mu FL, Li HY (2014) Power load classification based on spectral clustering of dual-scale. In: Proceedings of the IEEE international conference on control science and systems engineering. IEEE
Ye M, Liu Y, **ong W et al (2014) Voltage stability research of receiving-end network based on real-time classification load model. In: Proceedings of the IEEE international conference on mechatronics and automation, pp 1866–1870
Che JX, Wang JZ (2014) Short-term load forecasting using a kernel-based support vector regression combination model. Appl Energy 132(11):602–609
Liu N, Tang Q, Zhang J et al (2014) A hybrid forecasting model with parameter optimization for short-term load forecasting of microgrids. Appl Energy 129:336–345
Hernández L, Baladrón C, Aguiar JM et al (2014) Artificial neural networks for short-term load forecasting in microgrids environment. Energy 75(C):252–64
Nickel M, Murphy K, Tresp V et al (2015) A review of relational machine learning for knowledge graphs. Proc IEEE 104(1):11–33
Drumond LR, Diaz-Aviles E, Schmidt-Thieme L et al (2014) Optimizing multi-relational factorization models for multiple target relations. In: Proceedings of the ACM international conference, pp 191–200
Schweppe F, Wildes J (1970) Power system static-state estimation, part I, II, III. IEEE Trans Power Appl Syst PAS-89(1):120–35
Xu X, He X, Ai Q et al (2015) A correlation analysis method for power systems based on random matrix theory. IEEE Trans Smart Grid
Yang Q (2016) Cross-domain data fusion. Computer 49(4):18–18
Diou C, Stephanopoulos G, Panagiotopoulos P et al (2011) Large-scale concept detection in multimedia data using small training sets and cross-domain concept fusion. IEEE Trans Circuits Syst Video Technol 20(12):1808–1821
Tsai CC, Chang-Chien LR, Chen IJ et al (2016) Practical considerations to calibrate generator model parameters using phasor measurements. IEEE Trans Smart Grid 1–11
Mishra DP, Samantaray SR, Joos G (2016) A combined wavelet and data-mining based intelligent protection scheme for microgrid. IEEE Trans Smart Grid 7(5):2295–2304
Hana S (2012) SAP AG, in-memory database, computer appliance. Vertpress
Kennedy SJ (2012) MCP. M.C.P, Massachusetts institute of technology transforming big data into knowledge: experimental techniques in dynamic visualization. Massachusetts Inst Technol
Cuzzocrea A (2014) Privacy and security of big data: current challenges and future research perspectives. In: Proceedings of the international workshop on privacy and security of big data. ACM, pp 45–47
Bertino E (2015) Big data—security and privacy. In: Proceedings of the IEEE international congress on big data. IEEE
Chiang, M, Zhang T (2016) Fog and IoT: an overview of research opportunities. IEEE Internet of Things 3(6):854–864
Pan J, McElhannon J (2018) Future edge cloud and edge computing for internet of things applications. IEEE Internet of Things J 5(1):439–449
Chen A et al (2017) HCOS: a unified model and architecture for cloud operating system. ZTE Commun Mag 15(4):23–29
Shi W et al (2016) Edge computing: vision and challenges. IEEE Internet of Things J 3(5(Oct.)):637–646
Pan J, McElhannon J (2018) Future edge cloud and edge computing for internet of things applications. IEEE Internet Things J 5(1):439–449
Chen S et al (2017) Fog computing. IEEE Internet Comput 21(2):4–6
Bonomi F, Milito R, Natarajan P et al (2014) Fog computing: a platform for internet of things and analytics. In: Bessis N, Dobre C (eds) Big data and internet of things: a roadmap for smart environments, studies in computational intelligence, vol 546. Springer, Cham
Dolui K, Datta SK (2017) Comparison of edge computing implementations: fog computing, cloudlet and mobile edge computing. In: IEEE Global Internet of Things summit (GIoTS), Geneva, 1–6
Shi W et al (2016) Edge computing: vision and challenges. IEEE Internet Things J 3(5):637–646
Aazam M, Huh EN (2016) Fog computing: the cloud-IOT/IoE middleware paradigm. IEEE Potentials 35(3):40–44
Hu P et al (2017) Survey on fog computing: architecture, key technologies, applications and open issues. J Netw Comput Appl (Nov.):27–42
Open Fog Consortium, OpenFog reference architecture for fog computing. https://www.openfogconsortium.org/wp-content/ploads/OpenFog_Reference_Architecture_2_09_17-FINAL-1.pdf
Swiatek P et al (2012) Service composition in knowledge-based SOA systems. N Gener Comput 30(2):165–188
Gubbi J et al (2013) Internet of Things (IoT): a vision, architectural elements, and future directions. Future Gener Comput. Syst 29(7(Sept.)):1645–1660
Silva DM et al (2016) Villalba. Wirel Pers Commun 91, 1711
Jara AJ et al (2013) Mobile discovery: a global service discovery for the internet of things. In: 27th IEEE international conference on advanced information networking and applications workshops, Barcelona, pp 1325–1330
Stelmach P (2013) Service composition scenarios in the internet of things paradigm, DoCEIS 2013. IFIP Adv Inf Commun Technol 394
Rykowski J et al (2014) Comparison of architectures for service management in IoT and sensor networks by means of OSGi and REST services. In: 2014 Federated conference on computer science and information systems, Warsaw, pp 1207–1214
Lin N, Shi W (2014) The research on Internet of things application architecture based on web. In: 2014 IEEE workshop on advanced research and technology in industry applications (WARTIA), Ottawa, ON, pp 184–187
Freeman H, Zhang T (2016) The emerging era of fog computing and networking. IEEE Commun Mag 54(6):4–5
Yang Y (2017) FA2ST: fog as a service technology. In: 2017 IEEE 41st annual computer software and applications conference (COMPSAC), Turin, pp 708–708
Tran DH et al (2017) OaaS: offload as a service in fog networks. Computing 99:1081
Shi Y et al (2015) The fog computing service for healthcare. In: 2015 2nd IEEE international symposium on future information and communication technologies for ubiquitous healthcare (Ubi-HealthTech), Bei**g, pp 1–5
Sheng Z et al (2013) A survey on the ietf protocol suite for the internet of things: standards, challenges, and opportunities. IEEE Wirel Commun 20(6):91–98
Salman T, Jain R (2015) Networking protocols and standards for internet of things, internet of things and data analytics handbook. Wiley
Aijaz A et al (2015) CORPL: a routing protocol for cognitive radio enabled AMI networks. IEEE Trans Smart Grid 6(1(Jan.)):477–485
Basagni S et al (2012) Channel-aware routing for underwater wireless networks. In: 2012 Oceans—Yeosu, Yeosu, pp 1–9
Zhou Z et al (2016) E-CARP: an energy efficient routing protocol for UWSNS in the internet of underwater things. IEEE Sens J 16(11):4072–4082
IEEE Standard for a Convergent Digital Home Network for Heterogeneous Technologies Amendment 1: Support of New MAC/PHYs and Enhancements, IEEE Std 1905.1a-2014, pp 1–52
Wobschall D (2007) IEEE 1451-a universal transducer protocol standard. In: 2007 IEEE autotestcon, Baltimore, MD, pp 359–363
Moritz G et al (2011) A lightweight SOAP over CoAP transport binding for resource constraint networks. In: 2011 IEEE eighth international conference on mobile ad-hoc and sensor systems, Valencia, pp 861–866
Santos GLD et al (2015) A DTLS-based security architecture for the Internet of Things. In: 2015 IEEE symposium on computers and communication (ISCC), Larnaca, pp 809–815
Yassein MB et al (2016) Application layer protocols for the Internet of Things: a survey. In: 2016 International conference on engineering & MIS (ICEMIS), Agadir, pp 1–4
D’Angelo G et al (2016) Simulation of the Internet of Things. In: 2016 International conference on high performance computing and simulation (HPCS), Innsbruck, pp 1–8
Zhao Y et al (2015) NS3-based simulation system in heterogeneous wireless network. In: 11th International conference on wireless communications, networking and mobile computing (WiCOM 2015), Shanghai, pp, 1–6
Osterlind F et al (2006) Cross-Level sensor network simulation with COOJA. In: 31st IEEE conference on local computer networks, Tampa, FL, pp 641–648
Muscalagiu I et al (2013) Large scale multi-agent-based simulation using NetLogo for the multi-robot exploration problem. In: 2013 11th IEEE international conference on industrial informatics (INDIN), Bochum, pp 325–330
Zeng X et al (2017) IOTSim: a simulator for analysing IoT applications. J Syst Architect 72(issue C):93–107
Gupta H et al (2016) ifogsim: a toolkit for modeling and simulation of resource management techniques in internet of things, edge and fog computing environments, tech. report CLOUDS-TR-2016-2. Cloud Computing and Distributed Systems Laboratory, University of Melbourne
Mehdi K et al (2014) CupCarbon: a multi-agent and discrete event wireless sensor network design and simulation tool. In: Proceedings of the 7th international ICST conference on simulation tools and techniques (SIMUTools ’14), Brussels, Belgium, Belgium, pp 126–131
**an X et al (2008) Comparison of OMNeT++ and other simulator for WSN simulation. In: IEEE conference on industrial electronics and applications (ICIEA’08), pp 1439–1443
Ghayvat H et al (2015) Simulation and evaluation of ZigBee based smart home using Qualnet simulator. In: 2015 9th International conference on sensing technology (ICST), Auckland, pp 536–542
Burg A et al (2018) Wireless communication and security issues for cyber-physical systems and the internet-of-things. Proc IEEE 106(1):38–60
Suryani V, Selo, Widyawan (2016) A survey on trust in internet of things. In: 2016 8th International conference on information technology and electrical engineering (ICITEE), Yogyakarta, pp 1–6
Kim H, Lee EA (2017) Authentication and authorization for the internet of things. IT Prof 19(5):27–33
Li X et al (2017) An IoT data communication framework for authenticity and integrity. In: 2017 IEEE/ACM second international conference on internet-of-things design and implementation (IoTDI), Pittsburgh, PA, pp, 159–170
Margelis G et al (2016) Smart attacks on the integrity of the internet of things: avoiding detection by employing game theory. In: 2016 IEEE global communications conference (GLOBECOM), Washington, DC, pp, 1–6
Fotiou N et al (2016) Access control for the internet of things. In: 2016 International workshop on secure internet of things (SIoT), Heraklion, pp 29–38
Mrityunjaya DH et al (2017) Smart transportation. In: 2017 International conference on I-SMAC (IoT in social, mobile, analytics and cloud) (I-SMAC), Palladam, pp 1–5
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2023 Chemical Industry Press
About this chapter
Cite this chapter
Zeng, X., Bao, S. (2023). Big Data in Smart Grid and Edge Computing of the IoT. In: Key Technologies of Internet of Things and Smart Grid. Advanced and Intelligent Manufacturing in China. Springer, Singapore. https://doi.org/10.1007/978-981-99-7603-4_5
Download citation
DOI: https://doi.org/10.1007/978-981-99-7603-4_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-7602-7
Online ISBN: 978-981-99-7603-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)