Abstract
Service composition is an important and effective technique that enables atomic services to be combined together to forma more powerful service, i.e., a composite service. With the pervasiveness of the Internet and the proliferation of interconnected computing devices, it is essential that service composition embraces an adaptive service provisioning perspective. Reinforcement learning has emerged as a powerful tool to compose and adapt Web services in open and dynamic environments. However, the most common applications of reinforcement learning algorithms are relatively inefficient in their use of the interaction experience data, whichmay affect the stability of the learning process when deployed to cloud environments. In particular, they make just one learning update for each interaction experience. This paper introduces a novel approach that aims to achieve greater data efficiency by saving the experience data and using it in aggregate to make updates to the learned policy. The proposed approach devises an offline learning scheme for cloud service composition where the online learning task is transformed into a series of supervised learning tasks. A set of algorithms is proposed under this scheme in order to facilitate and empower efficient service composition in the cloud under various policies and different scenarios. The results of our experiments show the effectiveness of the proposed approach for composing and adapting cloud services, especially under dynamic environment settings, compared to their online learning counterparts.
Similar content being viewed by others
References
Achbany Y, Jureta I J, Faulkner S, Fouss F (2008). Continually learning optimal allocations of services to tasks. IEEE Transactions on Services Computing 1(3):141–154.
Al-Masri E, Q H Mahmoud (2007). Discovering the best web service. In Proceedings of WWW, 2007.
Al Ridhawi Y, Karmouch A (2015). Decentralized plan-free semantic-based service composition in mobile networks. IEEE Transactions on Services Computing 8(1):17–31.
Baladron C, Aguiar J M, Carro B, Calavia L, Cadenas A, Sanchez-Esguevillas A (2012). Framework for intelligent service adaptation to user’s context in next generation networks. IEEE Communications Magazine 50(3):18–25.
Chiu D, Agrawal G (2012). Cost and accuracy aware scientific workflow composition for service-oriented environments. IEEE Transactions on Services Computing 4(2):140–152.
Ding Z H, Jiang M Y, Kandel A (2012). Port-based reliability computing for service composition. IEEE Transactions on Services Computing 5(3):422–436.
Fernandez H, Tedeschi C, Priol T (2013). A chemistry-inspired workflow management system for a decentralised workflow execution. IEEE Transactions on Services Computing. DOI:https://doi.org/10.1109/TSC.2013.27.
Garcia Llinas G A, Nagi R (2015). Network and QoS-based selection of complementary services. IEEE Transactions on Services Computing 8(1):79–91.
Hang C W, Kalia A K, Singh M P (2012). Behind the curtain: Service selection via trust in composite services. In Proceedings IEEE ICWS, June, 2012.
Hatzi O, Vrakas D, Nikolaidou M, Bassiliades N, Anagnostopoulos D, Vlahavas I (2011). An integrated approach to automated semanticweb service composition through planning. IEEE Transactions on Services Computing. DOI:https://doi.org/10.1109/TSC.2011.20.
Jiang W, Hu S, Lee D, Gong S, Liu Z (2012). Continuous query for QoS-aware automatic service composition. In Proceedings of IEEE ICWS, June, 2012.
Lee C H, Hwang S Y, Yen I L (2012). A service pattern model for flexible service composition. In Proceedings of IEEE ICWS, June, 2012.
Lee J W 2001. Stock price prediction using reinforcement learning. In Proceedings of IEEE ISIE, 2001.
Li H, Dagli C H, Enke D (2007). Short-term stock market timing prediction under reinforcement learning schemes. In Proceedings of IEEE ADPRL, 2007.
Moustafa A, Zhang M (2012). Towards proactive web service adaptation. In Proceedings of CAISE, 2012.
Moustafa A, Zhang M (2013). Multi-objective service composition using reinforcement learning. In Proceedings of ICSOC, Lecture Notes in Computer Science, 2013.
Sim K M (2012). Agent based cloud computing. IEEE Transactions on Services Computing 5(4):564–577.
Sutton R S, Barto A G (1998). Introduction to Reinforcement Learning(1ed). MIT Press, Cambridge, MA, USA.
Tang H, Liu W, Zhou L (2012). Web service composition method using hierarchical reinforcement learning. In Proceedings of GCN.
Wada H, Suzuki J, Yamano Y, Oba K (2012). E3: A multi-objective optimisation framework for SLA-aware service composition. IEEE Transactions on Services Computing 5(3):358–372.
Wang H, Chen X, Wu Q, Yu Q, Zheng Z, Bouguettaya A (2014). Integrating on-policy reinforcement learning with multi-agent techniques for adaptive service composition. In Proceedings of ICSOC volume 8831 of Lecture Notes in Computer Science, Springer Berlin Heidelberg.
Wang H, Wang X (2013). A novel approach to large-scale services composition. In Proceedings of APWeb, 2013.
Wang H, Zhou X, Zhou X, Liu W, Li W, Bouguettaya A (2010). Adaptive service composition based on reinforcement learning. In Proceedings of ICSOC, 2010.
Watkins C (1989). Learning from delayed rewards. PhD thesis, Cambridge University, England.
Yu Q, Bouguettaya A (2013). Efficient service skyline computation for composite service selection. IEEE Transactions on Knowledge and Data Engineering 25(4):776–789.
Zhang H, Chai H, Zhao W, Melliar-Smith P M, Moser L E (2012). Trustworthy coordination ofweb services atomic transactions. IEEE Transactions on Parallel and Distributed Systems 23(8):1551–1565.
Zheng Z, Hao M, Lyu M R, King I (2011). QoS-aware web service recommendation by collaborative filtering. IEEE Transactions on Services Computing 4(2):140–152.
Acknowledgments
The author thanks the reviewers for their valuable time and constructive feedbacks. This work has been supported by KAKENHI Grant Number 20288837.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Moustafa, A. On Learning Adaptive Service Compositions. J. Syst. Sci. Syst. Eng. 30, 465–481 (2021). https://doi.org/10.1007/s11518-021-5498-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11518-021-5498-0