Scalable, Secure, Fail Safe, and High Performance Architecture for Storage, Analysis, and Alerts in a Multi-site Landslide Monitoring System

Abstract

Wireless sensor networks can be deployed in landslide prone areas to monitor various geological and weather properties to detect a possible landslide and provide early warning to local public for evacuation. Implementing and managing a system for capturing sensor data from the deployment sites and transferring to the central database for storage, analysis, retrieval and prediction is an endeavor riddled with both natural and technical challenges. The usual remoteness of the landslide prone areas result in power restrictions, bandwidth constraints and frequent connectivity issues. As the sensors and systems are exposed to constant natural elements, they are prone to frequent failures and calibration issues. Our high performance, scalable, robust, and secure system; featuring multi-site landslide data capture, replication, storage, monitoring, and processing functionalities, surmounts all these challenges effectively. The scalability and performance is achieved by real-time streaming of compressed data, in-memory processing, bulk storage, and retrieval through partitioned tables. The security is achieved through authenticating and encrypting streamed data and kee** only minimal raw data on site. The fail-safety is achieved through automated reconnection, and persisting and cross-tracking data at each processing step. Finally the high performance in analysis and alerts are achieved by series of hierarchical and temporal aggregate tables. In this paper we present the architecture and features of our landslide data system along with the performance testing statistics and related analysis. We demonstrate that our system is capable of handling data from 100 sites, each having 1000 sensors and sending data once a minute using a single cloud server.