Abstract
Cloud environments reduce data center operating costs through resource sharing and economies of scale. Infrastructure-as-a-Service is one example that leverages virtualization to share infrastructure resources. However, virtualization is often insufficient to provide Software-as-a-Service applications due to the need to replicate the operating system, middleware and application components for each customer. To overcome this problem, multi-tenancy has emerged as an architectural style that allows to share a single Web application instance among multiple independent customers, thereby significantly improving the efficiency of Software-as-a-Service offerings. A number of platforms are available today that support the development and hosting of multi-tenant applications by encapsulating multi-tenancy specific functionality. Although a lack of performance guarantees is one of the major obstacles to the adoption of cloud computing, in general, and multi-tenant applications, in particular, these kinds of applications and platforms have so far not been in the focus of the performance and benchmarking community. In this paper, we present an extended version of an existing and widely accepted application benchmark adding support for multi-tenant platform features. The benchmark is focused on evaluating the maximum throughput and the amount of tenants that can be served by a platform. We present a case study comparing virtualization and multi-tenancy. The results demonstrate the practical usability of the proposed benchmark in evaluating multi-tenant platforms and gives insights that help to decide for one sharing approach.
Chapter PDF
Similar content being viewed by others
References
TPC Benchmark W, Transaction Processing Performance Council (2002)
Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R.H., Konwinski, A., Lee, G., Patterson, D.A., Rabkin, A., Stoica, I., Zaharia, M.: Above the clouds: A berkeley view of cloud computing. Tech. Rep. UCB/EECS-2009-28, EECS Department, University of California, Berkeley (February 2009)
Binnig, C., Kossmann, D., Kraska, T., Loesing, S.: How is the weather tomorrow?: towards a benchmark for the cloud. In: Proceedings of the Second International Workshop on Testing Database Systems (2009)
Bitcurrent. Bitcurrent cloud computing survey 2011. Tech. rep., bitcurrent (2011)
Cain, H.W., Rajwar, R., Marden, M., Lipasti, M.H.: An architectural evaluation of Java TPC-W. In: Proceedings of the Seventh International Symposium on High-Performance Computer Architecture (2001)
Chong, F., Carraro, G., Wolter, R.: Multi-tenant data architecture (June 2006)
Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with ycsb. In: Proceedings of the 1st ACM Symposium on Cloud Computing, SoCC 2010, New York, NY, USA (2010)
Eclipse Foundation. Eclipselink/development/indigo/multi-tenancy (October 2012)
Fehling, C., Leymann, F., Mietzner, R.: A framework for optimized distribution of tenants in cloud applications. In: 2010 IEEE 3rd International Conference on Cloud Computing, CLOUD (2010)
Google. Google Cloud Platform (November 2012), https://cloud.google.com/index
Guo, C.J., Sun, W., Huang, Y., Wang, Z.H., Gao, B.: A framework for native multi-tenancy application development and management. In: E-Commerce Technology and the 4th IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services (2007)
Herndon, B., Smith, P., Roderick, L., Zamost, E., Anderson, J., Makhija, V., Herndon, B., Smith, P., Zamost, E., Anderson, J.: Vmmark: A scalable benchmark for virtualized systems. Tech. rep., VMware (2006)
Huang, S., Huang, J., Dai, J., **e, T., Huang, B.: The hibench benchmark suite: Characterization of the mapreduce-based data analysis. In: ICDE Workshops (2010)
Islam, S., Lee, K., Fekete, A., Liu, A.: How a consumer can measure elasticity for cloud platforms. In: Proceedings of the Third Joint WOSP/SIPEW International Conference on Performance Engineering, New York, NY, USA (2012)
Kiefer, T., Schlegel, B., Lehner, W.: MulTe: A multi-tenancy database benchmark framework. In: Nambiar, R., Poess, M. (eds.) TPCTC 2012. LNCS, vol. 7755, pp. 92–107. Springer, Heidelberg (2013)
Koziolek, H.: Towards an architectural style for multi-tenant software applications. In: Proc. Software Engineering (SE 2010). LNI, vol. 159 (February 2010)
Koziolek, H.: The sposad architectural style for multi-tenant software applications. In: Proc. 9th Working IEEE/IFIP Conf. on Software Architecture (WICSA 2011), Workshop on Architecting Cloud Computing Applications and Systems (July 2011)
Krebs, R., Momm, C., Kounev, S.: Architectural Concerns in Multi-Tenant SaaS Applications. In: Proceedings of the 2nd International Conference on Cloud Computing and Services Science, CLOSER 2012 (2012)
Mell, P., Grance, T.: The NIST definition of cloud computing. digital (2011)
Momm, C., Krebs, R.: A Qualitative Discussion of Different Approaches for Implementing Multi-Tenant SaaS Offerings. In: Proceedings of Software Engineering 2011 (SE 2011), Workshop (ESoSyM 2011) (2011)
Natis, Y.: Gartner reference model for elasticity and multitenancy. Gartner report, Gartner (June 2012)
Packman, E., Taylor, P., Rachitsky, L., Rejali, S., Power, S., Rae, I., Koffler, D.: Bitcurrent: Cloud comuting performance. Tech. rep., bitcurrent (June 2010)
SAP AG. SAP NetWeaver Cloud (November 2012), https://netweaver.ondemand.com
Smith, D.: Hype cycle for cloud computing. Tech. rep., Gartner, ID Number: G00214915 (July 2011)
Spec. Specjenterprise2010, (November 2012), http://www.spec.org/jEnterprise2010/
Wang, W., Huang, X., Qin, X., Zhang, W., Wei, J., Zhong, H.: Application-level cpu consumption estimation: Towards performance isolation of multi-tenancy web applications. In: IEEE CLOUD (2012)
Wang, Z.H., Guo, C.J., Gao, B., Sun, W., Zhang, Z., An, W.H.: A study and performance evaluation of the multi-tenant data tier design patterns for service oriented computing. In: IEEE International Conference on e-Business Engineering, ICEBE 2008 (2008)
Weissman, C.D., Bobrowski, S.: The design of the force.com multitenant Internet application development platform. In: Proceedings of the 35th SIGMOD International Conference on Management of Data, SIGMOD 2009. ACM (2009)
Zhang, Y., Wang, Z., Gao, B., Guo, C., Sun, W., Li, X.: An effective heuristic for on-line tenant placement problem in saas. In: IEEE International Conference on Web Services (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Krebs, R., Wert, A., Kounev, S. (2013). Multi-tenancy Performance Benchmark for Web Application Platforms. In: Daniel, F., Dolog, P., Li, Q. (eds) Web Engineering. ICWE 2013. Lecture Notes in Computer Science, vol 7977. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39200-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-642-39200-9_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39199-6
Online ISBN: 978-3-642-39200-9
eBook Packages: Computer ScienceComputer Science (R0)