The Tera MTA combines a uniform shared memory programming model, implemented using fine-grain multithreaded processors and a high bandwidth network, with an optimizing compiler that removes most of the drudgery from high performance parallel programming. The compiler and the architecture were designed together with highly satisfactory results. This talk will review and assess some of the decisions, mostly good but a few bad, that we made along the way.
Burton Smith is Chairman and Chief Scientist of Tera Computer Company. He received the BSEE from the University of New Mexico in 1967 and the Sc.D. from MIT in 1972. From 1985 to 1988 he was Fellow at the Supercomputing Research Center of the Institute for Defense Analyses in Maryland. Before that he was Vice President, Research and Development at Denelcor, Inc. and was chief architect of the HEP computer system. Dr. Smith is Fellow of the ACM and of the IEEE, and won the IEEE-ACM Eckert-Mauchly award in 1991. His main interest is general purpose parallel computer architecture.
Although it offers great potential for computing from the desktop to the supercomputer, scalable parallelism has never quite lived up to its advance billing. What has delayed its success? The major impediment has been the difficulty of solving some key software problems. Foremost among these is the problem of support for architecture-independent parallel programming. Software developers need to be able to rely on the retargetability of their codes for new parallel machines as they emerge. The absence of such assurance has been the principal reason for the slow acceptance of scalable parallelism to date.
Now, however, satisfactory solutions are emerging. For scientific problems, the most widely used solution is Message Passing Interface (MPI) which provides a standard interface to message-passing libraries, which can be efficiently implemented on all varieties of scalable machines, including shared-memory systems. However, programming in MPI is not easy-it requires that all of the details of communication be handled explicitly by the programmer.
As an example of an alternative approach, I will discuss High Performance Fortran (HPF), which is designed to support the construction of architecture-independent data-parallel programs. HPF allows the specification of machine-independent data parallel programs that can be compiled without change for many different parallel computer architectures. Although over twenty companies have developed or are working on products related to High Performance Fortran, it has not yet been an unqualified success. In this talk I will review the problems that HPF has faced and discuss technical solutions that my research group and the HPF community have been developing. These technologies are now beginning to find there way into commercial products that make the use of HPF more attractive than ever. The result has been a number of new applications, some over 100,000 lines in length.
The talk will conclude with a summary of the lessons learned from HPF and their implications for supporting machine-independent parallel programming on emerging new architectures and distributed computing configurations. These architectures present many new and interesting challenges for the compiler writer, the library developer, the operating system implementer, and the machine designer.
Short Biographical Sketch: Ken Kennedy received a B.A. in mathematics from Rice University in 1967, an M.S. in mathematics from New York University in 1969, and a Ph.D. in computer science from New York University in 1971. Since 1971, he has been a faculty member at Rice University, first in the Department of Mathematical Sciences and then in the Department of Computer Science, which he founded in 1984. Professor Kennedy served as Chair of the Department of Computer Science from 1984 until 1988 and again from 1990 until 1992. Since 1985, he has held the Noah Harding Professorship in that department. He is currently Director of the Center for Research on Parallel Computation, an NSF Science and Technology Center with seven participating institutions-Rice University, California Institute of Technology, Los Alamos National Laboratory, Argonne National Laboratory, the University of Tennessee, Syracuse University, and the University of Texas at Austin. In 1997 he was appointed co-chair of the Presidential Advisory Committee on High Performance Computing and Communications, Information Technology, and Next Generation Internet.
Dr. Kennedy has published over one hundred technical articles and supervised more than thirty Ph.D. dissertations on programming support software for high-performance computer systems. He has supervised the construction of two substantial software systems for programming parallel machines: an automatic vectorizer for Fortran 77 and an integrated scientific programming environment. His current research focuses on extending techniques developed for automatic vectorization to programming tools for parallel computer systems and high-performance microprocessors. Through the Center for Research on Parallel Computation, he is seeking to develop new strategies for supporting architecture-independent parallel programming, especially in science and engineering. To that end, he chaired the High Performance Fortran Forum, an informal standardization group that defined a set of extensions to Fortran 90 for data parallel programming.
Professor Kennedy was elected to the National Academy of Engineering in 1990. He was named a fellow of the AAAS in 1994 and of the ACM and IEEE in 1995. In recognition of his achievements in compilation for high performance computer systems, he was named the recipient of the 1995 W. W. McDowell Award, the highest research award of the IEEE Computer Society.