Abstract
We consider the problem of execution time prediction for non-deterministic multi-phase bulk synchronous computations in multiprocessors. We characterize the computations in two stochastic workload evolution models: additive and multiplicative. The additive model reflects the commutations in which the workload changes between phases are independent of processes' present workload. The multiplicative model becomes relevant when the workload change in a process is proportional to its load base. We take advantage of their salient features and show that conventional approaches based on central limit theorem in statistics are viable to predict the execution time for long run computations. By an elegant coordination of results from order statistics and convergence rates in the central limit theorem, we derive tighter bounds on execution time of short run computations, under some mild assumptions on their workload change distributions. Accuracy of the predictions is analyzed rigorously and verified by simulations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. H.-S. Ang and W. H. Tang. Probability Concepts in Engineering Planning and Design, Vol. II. Rainbow Bridge, 1984.
R. Armstrong, D. Hensgen, and T. Kidd. The relative performance of various map** algorithms is independent of sizable variances in run-time predictions. In Proceedings of the 7th Heterogeneous Computing Workshop, 1998.
M. Colajanni, F. Presti, and S. Tucci. A hierarchical approach for bounding the completion time distribution of stochastic task graphs. Performance Evaluation, 41:1–22, 2000.
H. A. David. Order Statistics, 2nd ed. John Wiley, New York, 1981.
F.-T. Fong, C. Xu, and L. Wang. Optimal periodic remap** of bulk synchronous computations on multiprogrammed distributed systems. In Proceedings of the 14th IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 103–108, April 2000.
G. Fox, R. Williams, and P. Messina. Parallel Computer Works! Morgan Kaufmann, 1995.
G. C. Fox, M. A. Johnson, G. A. Lyzenga, S. W. Otto, J. K. Salmon, and D. W. Walker. Solving Problems on Concurrent Processors, Vol. I. Prentice Hall, Englewood Cliffs, NJ, 1988.
P. Hall. Rates of Convergence in the Central Limit Theorem. Pitman, London, 1982.
M. Iverson, F. Ozguner, and G. Follen. Run-time statistical estimation of task execution times for heterogeneous distributed computing. In Proceedings of the 1996 High Performance Distributed Computing Conference, pp. 263–270, 1996.
M. Iverson, F. Ozguner, and L. Potter. Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment. In Proceedings of the 1999 Heterogeneous Computing Workshop. IEEE Computer Society Press, Los Alamitos, Calif., 1999.
T. Kidd, D. Hensgen, L. Moore, R. Freund, D. Charley, M. Halderman, and M. Janakiraman. Studies in the useful predictability of programs in a distributed and homogeneous environment, 1995.
K. Li. Stochastic bounds for parallel program execution times with processor constraints. IEEE Transactions on Computers, 46:630–636, 1997.
S. Madala and J. B. Sinclair. Performance of synchronous parallel algorithms with regular structures. IEEE Transactions on Parallel and Distributed Systems, 2:105–116, 1991.
D. M. Nicol and P. F. Reynolds. Optimal dynamic remap** of data parallel computation. IEEE Transactions on Computers, 39:206–219, 1990.
D. M. Nicol and J. H. Saltz. Dynamic remap** of parallel computations with varying resource demands. IEEE Transactions on Computers, 37:1073–1087, 1988.
G. D. Peterson and R. D. Chamberlain. Beyond execution time: Expanding the use of performance models. IEEE Parallel and Distributed Technology, pp. 37–49, Summer 1994.
B. Reistad and D. Gifford. Static dependent costs for estimating execution time. In Proceedings of the ACM International Conference on LISP and Functional Programming, pp. 65–78, 1994.
J. Robinson. Some analysis techniques for asynchronous multiprocessor algorithms. IEEE Transactions on Software Engineering, 5:24–31, 1979.
J. Yang, I. Ahmad, and A. Ghafoor. Estimation of execution times on heterogeneous supercomputer architecture. In Proceedings of the International Conference on Parallel Processing, pp. 219–226, 1993.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Xu, CZ., Wang, L.Y. & Fong, NT. Stochastic Prediction of Execution Time for Dynamic Bulk Synchronous Computations. The Journal of Supercomputing 21, 91–103 (2002). https://doi.org/10.1023/A:1013539532035
Issue Date:
DOI: https://doi.org/10.1023/A:1013539532035