Volume 7, Number 1 - Grid Computing
Volume 7, Number 1 - Grid Computing
By Michael Osias, IBM
Grid computing is distributed computing over the network enabled by open standards. It is the next evolutionary step in distributed computing resulting from the availability and convergence of capabilities offered by other distributed technology such as Common Object Request Broker Architecture (CORBA), distributed computing environment (DCE), Java 2 Enterprise Edition (J2EE), Web services, and the build out of network capacity during the late 90s. The increasing computing demands and requirements that governments, business, and science and research are placing on information technology are driving the convergence of these distributed technologies to address these demands.
Initially, Grid computing was formed in the scientific and research communities to enable large scale computations whose capacity requirements exceeded any single organization's resources. Collaborating researchers needed a way to aggregate and share super computing resources and make them available to do their work. In essence, by linking these resources together using Grid resource sharing protocols, a larger, virtual super computer was formed composed of the shared resources. Often, these shared resources were comprised of heterogeneous compute and storage resources and tended to be geographically distributed. Increasingly, industry began to face the same pressures and needs of having to handle increased workload and data storage requirements, often within or between distributed enterprises, without additional capacity.
As these types of large scale shared resource complexes became more commonplace, efforts to standardize the infrastructure and services began. Early leaders of the standardization efforts were from research, science, and academia. Industry and governments, searching for solutions to similar problems, recognized the emerging Grid technology as a viable computing technology to solve problems today and provide a clear evolutionary path to the future. As adoption grew, so did the participation in the development of the standards, which today include participants from industry, government, and academia. As a result of this expanded adoption and participation, there is a rich set of Grid computing standards and technology.
Industry leaders are heavily involved in developing and refining standards. IBM in particular has played a significant role in the development of the standards and contributed to the development of the open source standards based software, the Globus Toolkit. IBM has incorporated Grid technology across all lines of business including services, hardware, and software, and is delivering over 100 leading edge production Grid projects around the world.
Grid computing is a parallel and distributed architecture that enables the sharing, selection, and aggregation of geographically distributed resources. Specific kinds of resources applicable to the Department of Defense (DoD) environment include computers (PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDAs, etc), software (e.g., Service Providers vending content and expensive special purpose applications on demand), catalogued data and databases (e.g. transparent access to unit readiness, weapons characteristics databases etc.), special devices and sensors (e.g., reconnaissance satellites, UAVs, etc.). Additionally, Grid technology addresses resource availability, capability, costs, and user Quality of Service (QoS) requirements. Grid infrastructure services, such as schedulers, resource managers, information services, and data services, are all essential to the Grid complex to provide inherent resiliency, resource optimization, and non-trivial quality of service while managing resources in the battlespace or DoD environment.
Traditionally, as a reflection of the origin of Grid technology, the usage model is oriented toward efficient use of all available processing horsepower. Grid standards are now leveraging innovations in Web services technology with Open Grid Services Architecture (OGSA), moving Grid technology toward a service-oriented computing model. This enables market-based utility computing and a new Net-Centric paradigm.
Transforming defense intranets into powerful, self-managing, virtual computers is a technical objective that Grid technology can enable. This large scale virtual computing environment, supported and enabled by a common Grid computing substrate, enables a processing environment for full Net-Centricity. It fits with the DoD Global Information Grid (GIG) vision, with DoD's current state consisting of a vast collection of heterogeneous systems, just beginning to share communications and computing resources on wide area networks. Soon the GIG will be moving to implement the GIG Enterprise Services (GES). These enterprise services provide ubiquitous service-oriented access to DoD data. A Grid complex supporting these data services means that they perform according to quality of service policies, are standards based, and universally accessible to users and other applications. Grid services technology allows rapid, efficient coupling of legacy transactions systems and will enable new warrior or business applications. In addition, Grid-enabled enterprise services will begin to address globally distributed computing issues.
As the momentum of Grid adoption continues to grow, we will see more domains and more Grids being built out within, and at times, between these domains. The proliferation of Grid technology is where the Grid standards are so important. To be able to join together Grids, whether the intent is to tap into another organization's Grid for collaboration and form an inter-Grid, to interface with an infrastructure provider to augment an existing infrastructure and dynamically increase capacity on demand, or if statewide Grids want to interconnect with other states, universities, or federal Grids, the open Grid standards enable Grid interconnectivity. As the technology matures and Grids continue to be built out, inter-Grids will become more commonplace and provide the technical and architectural substrate for the next generation of computing.
In addition to allowing organizations to share resources within or across organizational boundaries in an inter-Grid scenario, individuals or departments may want to make resources available to a larger Grid. Standards again are the enabler, and using standards-based Grid middleware infrastructure services, resources can easily be added to existing Grids and made available for use. There are many choices for this infrastructure software, from open source software such as the J2EE-based Globus Toolkit to commercial offerings such as the IBM Grid Toolbox and DataSynapse.
Web services technology provides a standard approach to separating interface from implementation details in an Internet-based distributed computing environment. By decoupling interface from implementation, classical Grid technology can be used to develop systems that can be composed and reconfigured dynamically at run time. Building on Web services technologies, the Open Grid Services Infrastructure (OGSI) specification defines Web service interfaces and behaviors for creating, managing, and exchanging information among stateful Web service instances. OGSI is comprised of an extension of the base Web services standards, which currently do not address the notion of stateful Web services instances, and the definition of a set of interfaces that describe and standardize requestor-service interactions. These interactions provide the basis for additional semantic interoperability not provided in the base Web service standards. In essence, these interfaces define the basis of a Web services programming model.
With the exception of the simplest data retrieval or computation services, a service needs to keep some kind of state to provide any value above and beyond a basic Web-based extensible markup language (XML) document processor or a remote procedure call (RPC) interface. Many services have a distinct state associated with every single client accessing them. A common example of such a service is a shopping application with the shopping cart unique for each client. An e-commerce business can be created simply by integrating and coordinating sets of services which include on demand supply and procurement, shipping and routing, and accounting and billing, as well as Web application infrastructure services such as a shopping cart, an online address book service, and payment processing. By using available services, like a generic shopping cart service, the company saves money by assembling these component services into higher level services and applications. There is no need for them to develop and manage code and infrastructure for basic features such as a shopping cart. The service provider that provides the shopping cart must be able to accommodate many different shopping carts from multiple customers. In addition to the obvious service data elements needed for state such as the shopping cart contents and session information, the service provider must also track activity and usage data for billing the appropriate organization (not necessarily the person using the shopping cart, but the e-commerce customer who has integrated the service into their online business).
Another example is that of a database query service. It is analogous to using a database command client, such as a DB2 command line interface, where each client has its own connection and consequently connection session state to the database server. This allows concurrent access to the database server with different clients performing a variety of database operations simultaneously. To extend the notion of a database connection to a services oriented architecture, a similar session and state management facility is required. Otherwise, using the classical Web services definition and forcing an analogy of a Web service as a singleton, the service is able to accommodate only a single data service client at a time, typically in the call pattern of a blocking call. Using the OGSI component programming model, each client can have its own instance of a service by either inspecting a Grid data service registry or by invoking the createService() port reference to instantiate a new service instance. With this model, multiple clients can interact with the database service with their own individual session state preserved. An implementation strategy for this kind of service could involve exposing stateful j2ee entity and session beans as ogsi-compliant Grid services to provide data access.
A reliable file transfer service, like the one provided with the open source Globus Toolkit, also requires the ability to maintain instances and state if it is to service more than a single client at any given time. The persistent factory service again is used to create a unique instance of the service for a client. For those familiar with software patterns, the factory pattern mentioned herein is very similar in concept. It is an OGSI base service whose purpose is to create instances of services, much like a static Java factory object would create well formed instances of specific objects using getInstance(). Once a service instance is created, the client can then interact with the service to provide such information as source and destination, and even register for periodic updates from the service instance. The service state is maintained by service data elements which are stored as part of the service instance. Each instance has its own set of service data which has different values. An example of the service data for the reliable file transfer service is the percentage of the file that has been transferred. A client may subscribe to this service data element using the OGSI notification framework to receive periodic updates about the amount of the file transferred. By having stateful service instances, potentially transient but also with a lifetime beyond that of the initial SOAP envelope processing, a Grid service can provide services to many different clients, and possibly offer different quality-of-service levels and corresponding implementations based on the priority of the user. As an example a Grid service may manage bandwidth allocation between two network endpoints based on the user level and priority, allocating a higher percentage of the link to the higher privileged user.
The OGSI programming model is designed to integrate key Grid technologies to create a distributed system framework to provide for the controlled management of the distributed and often long-lived state that is commonly required in sophisticated distributed applications. Fundamental Grid technologies include information services, resource management, data services, and security. The creation of Grid service instances using a persistent factory service enables the leveraging of the resources in terms of intelligent load balancing and service routing. A createService operation on a factory service may involve a number of steps before the service implementation is instantiated and a Grid Service Handle (GSH) returned to the client. For example, the factory service may intercept the instantiation of the service implementation to determine if a suitable instance already exists. It may also inspect the Grid infrastructure by using OGSA-managed resource services to determine the current load on a node, or consult an index information service to determine what suitable resources are available in the Grid complex. Ultimately, the factory may instantiate a new hosting environment on another node which is used to instantiate the service. A GSH for the service instance on the new node is registered in the index service and returned to the client. The factory service may have its own mechanisms for these processing cases, or may hand off to underlying resource management Grid infrastructure services such as brokers and schedulers. The factory service then becomes more of a coordinator of workflow across various Grid infrastructure services.
There are many resources such as IBM RedBooks and the
IBM DeveloperWorks website Grid Computing zone. Also available is
the IBM Grid Toolbox for Multiplatforms v3.0. This enhanced derivative
of Globus allows developers to easily install, configure, develop, and
manage Grid services. Installation, systems management, and
application development enhancements let developers focus on Grid
applications development and less on the Grid infrastructure itself.
Mr. Michael J. Osias is a Technical Architect for IBM Grid Computing Americas, with a focus on Department of Defense and other Government agencies. He has e-Business technical architecture experience in large e-Business infrastructures including portals, B2B, e-Commerce, and legacy integration and transformation. Michael has a background in telecommunications, treasury and finance operations, and defense industry. Prior to being a Grid Technical Architect, Michael was an e-Business Architect covering several telcom and manufacturing accounts, while conducting independent Grid Computing research and participating in the Global Grid Forum. He received an IBM Invention Achievement Award for patent submissions and technical disclosures in such areas as web search technology and distributed transaction processing. Also, he is an IBM Certified WebSphere Systems Expert. He has an educational background in Mathematics and Computer Science with the State University of New York and other higher education institutions.
Author Contact Information
Michael J. Osias
E-mail: [email protected]
Cell: (201) 230-4905
Those looking to become key players in Grid technology and
get started developing Grid services and Grid architectures should have
a background in J2EE and Web services technologies, as well as
distributed systems. It's a small step from there to Grid computing and Grid services.
 |
 |
 |