Volume 7, Number 1 - Grid Computing
Volume 7, Number 1 - Grid Computing
By Ian Foster and Lee Liming, Argonne National Laboratory
The Grid is a set of technologies that, when deployed across
an enterprise's resources, ease the creation of applications and infrastructure
that facilitate collaborative and
adaptive systems.1 In this article, we
will explore the vital role that standards play in establishing the Grid. We
also discuss the important role played by open software, in particular
the software developed within the Globus Alliance, a collaboration of
researchers, system architects, and software developers that has been pursuing
this goal for several years.
Developers of Grid systems can build on a range of established and proposed standards that have emerged from work on the Internet, the World Wide Web, and Web-based applications—technologies that, like the Grid, seek to support resource sharing and collaboration. However, a number of key capabilities must be added before interoperable Grids can be constructed. Much work is currently underway to develop specifications, implement solutions, and propose standards (or additions to existing standards) for Grid computing. A particular focus of these efforts is system monitoring and management, an area that has not seen the broad adoption of standard solutions analogous to the Internet and the World Wide Web.2 This area has instead seen an abundance of proprietary systems and specifications without any particular set of solutions becoming dominant.
Our work in the Globus Alliance has historically consisted of the following activities, each of which represents a key challenge in the establishment of a useful Grid.
- Identifying and refining our understanding of the fundamental problems shared by potential users of the Grid (those who would develop or use Grid-based applications)
- Developing and refining solutions (including both specifications and implementations) to these fundamental problems
- Applying these solutions to real problems in order to test their applicability
- Proposing and championing these solutions as standards in broader communities
- Expanding availability (implementations and support) and adoption (uses in many settings) for these standard solutions
Our approach to realizing the Grid vision creates an ongoing tension between the short-term (but ongoing) need to have working software that supports user requirements and the longer-term need to formulate and promote new solutions that "push the envelope" of capabilities. In these early stages of the "Grid era"—when research is ongoing and constantly changing—this tension is quite strong: the solutions are evolving quickly enough that working implementations cannot keep up without making some sacrifices in continuity and backwards-compatibility. Our challenge is to find a good balance that serves both needs reasonably well.
Another challenge faced by the Globus Alliance (and the Grid community at large) is the need to formulate its solutions and standards in ways that appeal to the larger IT industry and the market it serves. Failure to meet this challenge would result in Grid computing languishing as a niche market, almost certainly without the critical mass required to sustain itself over time. Until standard Grid solutions are incorporated into mass market infrastructure technologies (as the Internet and the Web have been integrated into every major computer system now sold—not to mention cellular telephones, televisions, automobiles, and other commodity products), the Grid will remain beyond the reach of most of the users it is intended to help.
The Grid vision requires protocols (and interfaces and policies) that are not only open and general-purpose but also standard. Indeed, we would argue that "Grids will be standards-based or not at all." It is standardization that allows potential collaborators to establish resource-sharing arrangements quickly and easily with any interested party. It is standardization that will allow organizations to establish resource-sharing contracts routinely for acquiring resources on demand, thus avoiding the need to build expensive data centers designed to handle peak loads that remain underutilized most of the time. Standard solutions can allow us to move away from today's plethora of balkanized, incompatible, non-interoperable distributed systems and toward a model where computing and data capabilities are available as standardized, interchangeable commodities.
The World Wide Web Consortium (W3C) has been highly successful at promoting a number of proposed standards in various stages of development that are commonly referred to as the "Web services" specifications.3 The bases for these standards are the HTTP protocol, the XML encoding format, the SOAP remote procedure call mechanism, and the WSDL language. Web services standards have already been adopted by every major provider in the IT industry, and it is clear that they are heading toward a long life and a ubiquitous presence in current and future IT systems.
While Web services standards meet many needs in industry and are highly popular, there are currently no established Web services standards for the interfaces used to manage, monitor, and interact with resources and with services that maintain persistent state. Thus, it is not possible to define tools that monitor, manage, troubleshoot, etc., diverse resources in standard ways: critical issues if one is to build large-scale systems. It is precisely these issues with which Grid computing is concerned.
Our current strategy, therefore, is to ensure the establishment of Web services-based standards in these areas, and the Globus Alliance's efforts are currently focused on contributing significantly to this work. Our goal is a convergence of new and existing Web services standards with the solutions developed by the Grid community. A further goal is the standardization of the resulting technology so that it becomes at least as accessible as current Web services technology.
Standards organizations with which we are working to formalize specifications for the Grid include the Global Grid Forum (GGF)4, the World Wide Web Consortium (W3C), the Organization for the Advancement of Structured Information Standards (OASIS)5, and the Internet Engineering Task Force (IETF).6
The Global Grid Forum has played a key role in developing and articulating the Open Grid Services Architecture (OGSA), which defines the Grid community's "guiding principles" for Web services/Grid convergence.7 The GGF Open Grid Service Infrastructure (OGSI) Working Group formalized the OGSI v1.0 specification in 2003, which was our first attempt to define Web services standards for Grid computing.8 The open source Globus Toolkit® 3.0 (released in mid-2003) contained the first OGSI implementation and was followed soon after by a number of independent implementations (see sidebar).9
We were disappointed (but not surprised) to learn subsequently that the Web services community was not satisfied with OGSI v1.0, due largely to a belief that OGSI v1.0 did not fit well enough with existing Web services architectural principles. In late 2003 and early 2004, a smaller group of Grid and Web services architects (leaders in their respective fields) met intensively to work out a compromise set of specifications, which was announced at the GlobusWORLD 2004 conference in January 2004. This set of specifications is known collectively as the Web Services Resource Framework, or WSRF.10 It is our belief, expressed by leaders of both the Web services and Grid communities, that WSRF is the much-desired convergence of Web services and Grid technologies and that it will lead to a body of widely-available, highly-interoperable Grid technology. Of course, work to finalize the now-drafted WSRF specifications and to gain formal acceptance and endorsement by one or more standards bodies is yet to be done. The Globus Toolkit 4.0, expected in mid-2004, will include a WSRF implementation.
While WSRF is undoubtedly a major milestone in the Grid's history, it is nevertheless only one of many standards-related activities undertaken by the Grid community and by the Globus Alliance. The IETF recently approved our X.509 Proxy Certificate Profile as a proposed standard. This action standardizes the format of the Grid "proxy" certificates used to support the Grid's essential single sign-on and delegation capabilities. The GGF recently accepted our specification of the GridFTP v1.0 protocol for high-bandwidth data transfer over wide area networks.
We have also been working in the Grid and Web services communities toward draft specifications for Data Access and Integration (DAI) in the GGF, a WS-Agreement specification in the W3C for establishing service agreements between service providers and consumers (e.g., reserved network bandwidth or compute node scheduling priorities), and replica location capabilities for data management systems. We have actively engaged ongoing standards work in other areas, making Grid-focused contributions to the development of the WSDL 2.0 (Web Service Description Language) specification, the WSDM (distributed web service management) specification, and the SAML (Security Assertion Markup Language - an XML-based framework for exchanging security information) and XACML (Extensible Access Control Markup Language - an XML specification for expressing policies for information access over the Internet) languages for encoding policy specifications.
In our view, the definition and widespread adoption of standard protocols and interfaces is currently the single most critical problem facing the Grid community. Fortunately, we are making good progress. On the standards side, we have the increasingly effective Global Grid Forum and representation in other IT standards organizations. We have broadly-endorsed efforts underway to define OGSA and WSRF, which enshrine Grid technology within the highly-successful Web services suite of products and solutions.
On the implementation side, seven years of experience and refinement have produced a widely used de facto standard, the open source Globus Toolkit. IBM, Microsoft, Platform, HP, Sun, Avaki, Entropia, United Devices, and other IT industry members have expressed strong support for OGSA and announced their own products and services based on OGSA standards. In time, we will be able to state that for an entity to be part of the Grid it must implement OGSA protocols and interfaces, just as to be part of the Internet an entity must speak IP (among other things). Both open source and commercial products will interoperate effectively in this heterogeneous, multi-vendor Grid world, thus providing the pervasive infrastructure needed to support compelling Grid applications.
Establishing the Grid and the standards that define it is still very much a "work in progress." Because we are taking a standards-based approach to the problem, it is open to input from many sources. The standards organizations (GGF, W3C, OASIS, IETF) all have open memberships, allowing organizations and individuals to become contributors to their work. The Globus Alliance also welcomes new partners in our broad mission to realize the Grid vision through the Globus Alliance Affiliates program (for academic and R&D members) and the Globus Alliance Commercial Affiliates program (for corporate participation). See www.globus.org for details.
1 I. Foster. What is the Grid? A Three Point Checklist. GRIDToday, July 20, 2002.
2 I. Foster, C. Kesselman, S. Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International J. Supercomputer Applications, 15(3), 2001.
3 World Wide Web Consortium (W3C), http://www.w3.org/. The W3C's activities with respect to Web services are detailed at http://www.w3.org/2002/ws/.
4 Global Grid Forum, http://www.grid-forum.org/.
5 Organization for the Advancement of Structured Information Standards (OASIS), http://www.oasis-open.org/.
6 Internet Engineering Task Force (IETF), http://www.ietf.org/.
7 I. Foster, C. Kesselman, J. Nick, S. Tuecke. The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration. Open Grid Service Infrastructure WG, Global Grid Forum, June 22, 2002.
8 S. Tuecke, K. Czajkowski, I. Foster, J. Frey, S. Graham, C. Kesselman, T. Maguire, T. Sandholm, P. Vanderbilt, D. Snelling. Open Grid Services Infrastructure (OGSI) Version 1.0. Global Grid Forum Draft Recommendation, 6/27/2003.
9 The Globus Toolkit software is described in detail and available for download at http://www-unix.globus.org/toolkit/. Globus Toolkit is a registered trademark held by the University of Chicago.
10 WSRF status and activities are described in detail at http://www.globus.org/wsrf/.
Lee Liming is the Manager of the Distributed Systems Laboratory at Argonne National Laboratory, the "Chicago home" of the Globus Alliance. His work with Ian Foster, Carl Kesselman, and Steve Tuecke on Grid computing began in 1999, and he is currently responsible for work resulting in the Globus Toolkit and related Grid technologies. Through his work at Argonne National Laboratory, Mr. Liming is engaged in a broad range of Grid deployment and application projects including NEESgrid, the NSF Middeware Initiative/GRIDS Center, the NASA Information Power Grid, and the National Computational Science Alliance. He was previously a product manager and software engineer at ProQuest Information and Learning and an IT manager at the University of Michigan's School of Information.
Having joined the Mathematics and Computer Science Division of Argonne National Laboratory in 1989 as a postdoctoral researcher, Ian Foster has risen to the rank of senior scientist and associate division director. Since 1996, he has held a joint appointment in the department of computer science at the University of Chicago, where he now has the rank of professor. His research specifically addresses scalable authentication and authorization technologies (the Grid Security Infrastructure), information discovery and dissemination (the Meta Directory Service), resource reservation and management, high-performance data transfer, and data replication. He directs the Argonne Distributed Systems Laboratory, which designs and develops the Globus Toolkit and standards on which it is based. The toolkit has been adopted as the central component in Grid solutions offered by IBM, HP, Oracle and other major IT companies.
Lee Liming
Argonne National Laboratory
E-mail: [email protected]
Ian T. Foster
Argonne National Laboratory
E-mail: [email protected]
 |
 |
 |