Abstract
This paper presents a simulation-based performance prediction framework for large scale data-intensive applications on large scale machines. Our framework consists of two components: application emulators and a suite of simulators. Application emulators provide a parameterized model of data access and computation patterns of the applications and enable changing of critical application components (input data partitioning, data declustering, processing structure, etc.) easily and flexibly. Our suite of simulators model the I/O and communication subsystems with good accuracy and execute quickly on a high-performance workstation to allow performance prediction of large scale parallel machine configurations. The key to effcient simulation of very large scale configurations is a technique called loosely-coupled simulation where the processing structure of the application is embedded in the simulator, while preserving data dependencies and data distributions. We evaluate our performance prediction tool using a set of three data-intensive applications.
This research was supported by the Department of Defense, Advanced Research Projects Agency and Office of Naval Research, under contract No. N66001-97-C-8534, by NSF under contracts #BIR9318183, #ACI-9619020 (UC Subcontract #10152408) and #CDA9401151, by ARPA under contract No. #DABT63-94-C-0049 (Caltech subcontract #9503), and by grants from IBM and Digital Equipment Corporation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
A. Acharya, M. Uysal, R. Bennett, A. Mendelson, M. Beynon, J. K. Hollingsworth, J. Saltz, and A. Sussman. Tuning the performance of I/O intensive parallel applications. In Proc. of IOPADS’96. ACM Press, May 1996.
R. Agrawal and J. Shafer. Parallel mining of association rules. IEEE Transactions on Knowledge and Data Engineering, 8(6):962–9, Dec. 1996.
R. Bagrodia, S. Docy, and A. Kahn. Parallel simulation of parallel file systems and I/O programs. In Proceedings of the 1997 ACM/IEEE SC97 Conference. ACM Press, Nov. 1997.
J. Brehm, M. Madhukar, E. Smirni, and L. Dowdy. PerPreT-a performance prediction tool for massively parallel systems. In Proceedings of the Joint Conference on Performance Tools / MMB 1995, pages 284–298. Springer-Verlag, Sept. 1995.
C. F. Cerco and T. Cole. User’s guide to the CE-QUAL-ICM three-dimensional eutrophication model, release version 1.0. Technical Report EL-95-15, US Army Corps of Engineers Water Experiment Station, Vicksburg, MS, 1995.
C. Chang, B. Moon, A. Acharya, C. Shock, A. Sussman, and J. Saltz. Titan: a high-performance remote-sensing database. In Proceedings of the 13th International Conference on Data Engineering, Apr. 1997.
M. J. Clement and M. J. Quinn. Using analytical performance prediction for architectural scaling. Technical Report TR BYU-NCL-95-102, Networked Computing Lab, Brigham Young University, 1995.
R. Ferreira, B. Moon, J. Humphries, A. Sussman, J. Saltz, R. Miller, and A. Demarzo. The Virtual Microscope. In Proc. of the 1997 AMIA Annual Fall Symposium, pages 449–453. American Medical Informatics Association, Oct. 1997.
A. Gürsoy and L. V. Kalé. Simulating message-driven programs. In Proceedings of Int. Conference on Parallel Processing, volume III, pages 223–230, Aug. 1996.
S. Herrod. Tango lite: A multiprocessor simulation environment. Technical report, Computer Systems Laboratory, Stanford University, 1993.
C. L. Mendes. Performance prediction by trace transformation. In Fifth Brazilian Symposium on Computer Architecture, Sept. 1993.
B. Moon and J. H. Saltz. Scalability analysis of declustering methods for multidimensional range queries. IEEE Transactions on Knowledge and Data Engineering, 10(2):310–327, March/April 1998.
S. Reinhardt, M. Hill, J. Larus, A. Lebeck, J. Lewis, and D. Wood. The Wisconsin Wind Tunnel: Virtual prototy** of parallel computers. In Proc. of the 1993 ACM SIGMETRICS Conf. on Measuring and Modeling of Computer Systems, 1993.
M. Rosenblum, S. Herrod, E. Witchel, and A. Gupta. Complete computer system simulation: The SimOS approach. IEEE Parallel and Distributed Technology, 3(4):34–43, Winter 1995.
J. M. Schopf. Structural prediction models for high-performance distributed applications. In Cluster Computing Conference, 1997.
J. Simon and J.-M. Wierum. Accurate performance prediction for massively parallel systems and its applications. In Proceedings of Euro-Par’96, volume 1124 of LNCS, pages 675–688. Springer-Verlag, Aug. 1996.
Y. Yan, X. Zhang, and Y. Song. An effective and practical performance prediction model for parallel computing on non-dedicated heterogeneous NOW. Journal of Parallel and Distributed Computing, 38:63–80, Oct. 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Uysal, M., Kurc, T.M., Sussman, A., Saltz, J. (1998). A Performance Prediction Framework for Data Intensive Applications on Large Scale Parallel Machines. In: O’Hallaron, D.R. (eds) Languages, Compilers, and Run-Time Systems for Scalable Computers. LCR 1998. Lecture Notes in Computer Science, vol 1511. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49530-4_18
Download citation
DOI: https://doi.org/10.1007/3-540-49530-4_18
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65172-7
Online ISBN: 978-3-540-49530-7
eBook Packages: Springer Book Archive