Next: 4 GAT Application Utility
Up: 7 <
Previous: 2 API Descriptors List
Contents
Subsections
3 GAT Application API Descriptions
This section defines the model elements.
Note that model elements are an
implementation and language-specific detail and are thus not
described here.
Advertisement Subsystem
The GATAdvertService provides an interface to a generic Grid meta
data service. Such an services is a persistent external repository
for any information which may be useful outside the application
itself. Such information MAY include for example
- file names and locations published by the application,
- port and protocol information for contacting the application
online,
- information about jobs started by the application.
For that purpose, the GATAdvertService allows one to annotate the
appropriate GATObjects (which have to the
GATAdvertisable interface) with arbitrary meta data, and to store
that information in a meta data directory. That directory is also
called Advertisement Directory, or Advert Directory (AD).
The meta data are considered to be a list of key value pairs,
whereby keys and values are Strings. More powerful meta data
schemes with structured and typed meta data elements seem useful
and possible, but are not specified at the moment.
The meta data directory the information are stored in consists of a
simple hierarchical namespace, where the nodes are tupels of an
absolute path in that namespace, the GATObject published there, and
the attached set of meta data.
The namespace of the advert directory resambles a standard file
system namespace. An absolute path is a String with a
leading delimiter ('/'), plus a number of '/' delimited Strings
(path elements or directories), plus a trailing
undelimited String (name). A relative path does
not have a leading delimiter, and refers to nodes relative to some
other (absolute or relative) path. Relative Paths can contain
".." as elements, which refer to the previous element of
the referred absolute path.
Examples:
/www.zib.de/visual/data/test.h5 # absolute path
visual/data/test.h5 # relative path
../data/test.h5 # relative path
test.h5 # name of entry
The GATAdvertService has a current working directory, by default
'/'. All relative paths and names are with respect to that
directory initially. Allowed pathneames are as in the POSIX
definition, but of arbitrary length.
While querying the GATAdvertService, all entries in a directory
are, recursively or non recursively, matched against a specified
set of Query Meta Data (QMD). QMD are MD as specified above, with
additional syntax for value elements which are interpreted during
execution of the query. By the use of POSIX regular
expressions [7], that syntax supports matching following
cases (examples are in this font):
- presence/absence of a key
key="comment" value="$^" # key is not present
- empty values for a key
key="comment" value="^$" # key is present with empty value
- arbitrary values for a key
key="comment" value=".*" # key can have any or no value
key="comment" value=".+" # key can have any value
- specific values for a key
key="comment" value="^text$" # key can have a specific value
- regular expressions for values for a key
key="comment" value=".*http://.*" # key matches a regex (contains http URL)
key="comment" value="\d+" # key matches a regex (is numeric)
- absolute paths of entries in namespace
key="GAT_PATH" value="/GAT/" # entries in subtree /GAT/
- relative paths of entries in namespace
key="GAT_PATH" value="home/" # entries in relative subtree home/
- names of entries in namespace
key="GAT_NAME" value="^test.*" # entries with name matching 'test*'
- types of GATObjects stored
key="GAT_TYPE" value="GATEndpoint" # entry object of type GATEndpoint
Key strings starting with GAT_ are reserved for internal
GAT specific AD entries. Entries with such keys cannot be used to
store meta data. Currently, the user can use GAT_TYPE,
GAT_PATH and GAT_NAMEfor queries. If paths
and names are not specified, all entries relative to the current
working directory of the Advert Service are searched.
For many use cases it seems useful to extend the query language,
for example to cover numerical comparision (<, ==, >), or
to compare dates ("May, 12, 2003" > "01/02/03").
Also, GAT does not prescribe the usage of a specific namespace.
For certain communities, and also in general, it may be useful to
specify a structure of the namespace, in order to avoid naming
collisions ("/tmp/test.txt").
Ann runs a server application A, which creates a
GATEndpoint Endpoint_A. Application A listens
on that endpoint for incoming client connections in order to serve
their requests. A advertises the GATEndpoint with the
GATAdvertService.
Bill runs a client application, B, which wants to
communicate with A. B queries the
GATAdvertService for information about suitable GATEndpoints.
Getting the path of such endpoints in the GATAdvertService
hierarchy, B is able to reconstruct the GATEndpoint, and
to connect to it.
Ann creates a GATEndpoint.
Ann creates a GATAdvertService.
Ann advertises the GATEndpoint and meta data.
Ann listens on the GATEndpoint and waits for
incoming client connections
Bill creates a GATAdvertService.
Bill searches the GATAdvertService for suitable
GATEndpoint entries
Bill selects one entry
Bill gets a GATEndpoint from that AD entry
Bill connects to that GATEndpoint (and hence to Ann)
----------------------------------------
--- Synchroneous case ------------------
----------------------------------------
Ann:
GATAdvertService A_AS = new GATAdvertService (...);
GATEndpoint A_EP = new GATEndpoint (...);
String PATH = new String ("/Ann/GAT/Endpoints/EP1");
GATable MD = new GATable (...)
MD.put ("name", "myEndpoint")
MD.put ("protocol", "https")
MD.put ("application", "A")
MD.put ("owner", "Ann_1")
MD.put ("capability", "test")
AS.add (PATH, A_EP, MD);
GATPipe A_Pipe = A_EP.listen ();
// read/write on pipe - communicate with B
Bill:
GATAdvertService B_AS = new GATAdvertService (...);
B_AS.setPWD ("/Ann/GAT/");
bool RECURSION = True;
GATTable QUERY = new GATTable ()
QUERY.put ("GAT_TYPE", "GATEndpoint")
QUERY.put ("protocol", "https")
QUERY.put ("application", ".+")
QUERY.put ("owner", "Ann_\d+")
QUERY.put ("capability", "test.*")
list<String> PATHS = B_AS.find (QUERY, RECURSION);
GATEndpoint B_EP = B_AS.GetAdvertiseable (PATHS[0]);
GATPipe B_Pipe = B_EP.connect ();
// read/write on pipe - communicate with A
GATAdvertisable
An which is by any which wishes to get
advertized.
realizing this interface need to provide a
which takes a serialization of the (a String), and creates
a from the state information stored in that
serialization. That is effectively the only way to de-serialice a
GATAdvertisable .
Apart from the , the only for this interface
is Serialize, which takes care of the encapsulation of the state
of a GATAdvertiseable in a String. Though this method
will likely not be used by the majority of application programmers
directly, it will be used by adaptor writers who provide the
functionality of a GATAdvertService.
This takes the serialized state of an
GATAdvertisable , and re-creates that .
Inputs:
- GATContext -- context -- GATContext to be used for the
recreated .
- String -- serial -- Contains the 's serialization.
This obtains the serialized state encapsulated
by this instance.
Outputs:
- String -- serial -- Contains this instance's
serialization.
GATAdvertServiceGATObjectGATMonitorable
The GATAdvertService allows GATAdvertisable
to get published to and queried in an
advert directory. Such an advert directory is a meta data
directory with an hierarchical namespace attached.
In this use case the client wishes to advertise an
GATAdvertisable using a
GATAdvertService. The client first creates a String
describing the path at which they wish to advertise the
GATAdvertisable instance. Next the client creates
an of the GATTable and populates
it with name/value pairs which are to be the meta-data associated
with the GATAdvertisable instance. The client
then creates an of the GATAdvertService
and calls the Add on this
instance. This process is diagrammed in the following
sequence diagram ![[*]](crossref.png) :
Figure:
A sequence diagram for an of the
GATAdvertService.
|
[width=]AdvertServiceSequence1
|
In this use case the client wishes to find the path of
GATAdvertisable in the GATAdvertService. The
client first creates GATTable and populates it
with name/value pairs which are to be the meta-data associated
with the GATAdvertisable they wish to find. The
client then creates an of the GATAdvertService
and calls the Find on this
instance. This process is diagrammed in the following
sequence diagram :
Figure:
A sequence diagram for an of the
GATAdvertService.
|
[width=]AdvertServiceSequence2
|
Creates an of the GATAdvertService
corresponding to the passed GATContext.
Inputs:
- GATContext -- context -- Used to broker resources.
- GATPreferences -- preferences -- User preferences.
(OPTIONAL)
Destroyes the instance of GATAdvertService.
Add an Advertizable and related meta data to
the GATAdvertService, at path (absolute or relative to PWD). If
an GATAdvertService entry exists at the specified path, that
entry gets overwritten, and a warning is issued.
Inputs:
- GATAdvertiseable -- advert -- to be
entered into the GATAdvertService.
- GATTable -- metaData -- Meta data to be associated with
the passed GATAdvertiseable.
- String -- path -- Path (relative to PWD) of the new entry.
Deletes the specified entry from the GATAdvertService.
Inputs:
- String -- path -- Path of the entry to be deleted.
Get the meta data for the specified AD entry. The returned meta
data can be destroyed at any time.
Inputs:
- String -- path -- Path of the entry of interest.
Outputs:
- GATTable -- metaData -- Meta data of the entry of interest.
Get the Advertizable for the specified
GATAdvertService entry. The returned instance can be destroyed
at any time.
Inputs:
- String -- path -- Path of the entry of interest.
Outputs:
- GATAdvertiseable -- advert -- for
the entry of interest.
Query the GATAdvertService for entries matching the specified set
of meta data. The returned paths can be destroyed at any time.
Inputs:
- GATTable -- metaData -- Meta data describing the entries
to be searched for.
Outputs:
- List of Strings -- paths -- Paths, each pointing to
a matching entry.
The following extensions provides a more directory oriented
access scheme to the AD.
Specify the element of the GATAdvertService namespace to be used
as reference for relative paths.
Inputs:
- String -- path -- New absolute or relative reference path.
Returns the current element of the GATAdvertService namespace
used as reference for relative paths.
Outputs:
- String -- path -- Absolute reference path.
File and Streaming Subsystem
An application exchanging data and information via data streams
can use the GAT Streaming System. The system inherits from the
known concepts of BSD sockets, but simplifies that interface to
enable independent implementation.
The principal concept in GAT is to create an instance of
GATEndpoint, and (as server) to listen on it for incoming
connections, or (as client) connect to it. Both operations result
in an instance of a GATPipe, which can be used for two sided
communication. A third of the system,
GATFileStream, provides approximately the same
for remote file access.
In order to motivate the design for the GATEndPoint and GATPipe
objects, a simple use case is listed here.
Ann creates a communication endpoint (A) and advertises it. Ann
can further perform a listen on the endpoint, and gets a GATPipe
(pipe_A) on success.
Bill retrieves an advertisement from an GATAdvertService and uses
it to construct a communication endpoint (GATEndpoint B), and
tries to connect to A - he gets a GATPipe pipe_B. A connection
is established between pipe_A and pipe_B.
The advertisment is still valid and can be used by another party
to try to establish a connection with Ann.
Ann creates a GATEndpoint.
Ann uses the GATEndpoint to create a new GATPipe (A)
by calling listen on endpoint (callback).
Ann advertises the GATEndpoint.
Bill gets the GATEndpoint from a Advert Directory.
Bill creates a new GATPipe (B) using the GATEndpoint by calling
connect on it.
Ann and Bill can now write down their GATPipes
(A and B respectively) or read data from them.
Ann can continue to listen.
Cees does the same as Bill, and gets a third pipe.
----------------------------------------
--- Synchroneous case ------------------
----------------------------------------
Ann:
GATEndpoint A = new GATEndpoint (...)
GATAdvertService AS = new GATAdvertService (...)
GATPipe pipe_A1 = A.listen (...)
AS.add (A, ...)
pipe_A1.read (...)
pipe_A1.write (...)
delete (pipe_A1);
GATPipe pipe_A2 = A.listen (...)
pipe_A2.read (...)
pipe_A2.write (...)
delete (pipe_A2);
delete (A);
Bill:
GATAdvertService AS = new GATAdvertService (...)
list<String> paths = AS.query (...)
GATEndpoint B = AS.getAdvertizable (paths[0])
GATPipe pipe_B = B.connect ()
pipe_B.write (...)
pipe_B.read (...)
delete (pipe_B);
delete (B);
Cees:
GATAdvertService AS = new GATAdvertService (...)
list<String> paths = AS.query (...)
GATEndpoint C = AS.getAdvertisable (paths[0])
GATPipe pipe_C = B.connect ()
pipe_C.write (...)
pipe_C.read (...)
delete (pipe_C);
delete (C);
----------------------------------------
--- Asynchroneous case -----------------
----------------------------------------
Ann listens with GATPipeListener, and then uses the GAT event
servicing mechanism to serve incoming pipes.
Ann:
GATEndpoint A = new GATEndpoint (...)
GATPipeListener listen_A = new my_GATPipeListener (...)
GATAdvertService AS = new GATAdvertService (...)
AS.add (A, ...)
A.listen (listen_A, ...)
// do work (callback handling)
delete (A);
delete (listen_A);
sub GATPipeListener:ProcessPipe (GATPipe pipe_A)
{
pipe_A.read (...)
pipe_A.write (...)
pipe_A.close (...)
}
Bill:
GATAdvertService AS = new GATAdvertService (...)
list<String> paths = AS.query (...)
GATEndpoint B = AS.getAdvertisable (paths[0])
GATPipe pipe_B = B.connect ()
pipe_B.write (...)
pipe_B.read (...)
delete (pipe_B);
delete (B);
Depending on demand, it may be useful to extend the present
Stream API. For example, the ability to perform striped
or multiplexed reading/writing to collections of streams may be
useful. Such things can be implemented on top of the present
spec, and are to be defined separately.
GATStreamable
GATStreamable is a which provides
for connections to external entities such as files or other .
To send data down a GATStreamable it is necessary to construct a
Buffer, and pack it with data. Similarly to receive data a
buffer must be created in which the data will be stored. Sends
and receives MAY either be blocking, or asynchronous.
Asynchronous sends or receives must be completed by an
appropriate call.
The method of setting up a connection is the responsibility of the
(e.g. GATFileStream or GATPipe.)
Reads from this GATStreamable into the given Buffer.
This is a blocking call and only returns if the buffer
is full or if an error occurred. On error, the buffer MAY
contain partially read data.
Inputs/Outputs:
- Buffer -- buffer -- Buffer into which data are to be transferred
Outputs:
- Integer -- nbytes -- Number of bytes successfully read
into the buffer.
Reads from this GATStreamable into the given Buffer, but
returning immediately. The buffer SHOULD NOT be modified by the
application until a ReadFinish has been called on this
GATStreamable. The current position of the Buffer is advanced to
after the last item read.
Only one IRead MAY be active at once on a given GATStreamable.
This is a non-blocking call.
Inputs/Output:
- Buffer -- buffer -- Buffer into which data are to be transferred.
This finishes the current IRead call on this GATStreamable. All
data received between posting the IRead and this call will be
available in the associated Buffer on return.
Outputs:
- Bool -- gotData -- True if data were received, False if no data available.
- Integer -- nbytes -- Number of bytes successfully read
into the buffer.
This tests if data has been received in the current IRead.
Outputs:
- Bool -- canRead -- True if data can be received, False if no data available.
Writes data from the given Buffer through the GATStreamable. The
buffer is not modified.
This is a blocking call and only returns when all the data in
the buffer has been sent, or if an error occurs. On error, the
number of bytes written is returned.
Inputs:
- Buffer -- buffer -- Buffer from which data are to be
transferred.
Outputs:
- Integer -- nbytes -- Number of bytes successfully written.
Writes data from the given Buffer through the GATStreamable, but
returns immediately. The buffer SHOULD NOT be modified by the
application until a WriteFinish has been called on this
GATStreamable. The Buffer is not modified. The current
position the Buffer is advanced to the first unwritten item in
the Buffer. This is a non-blocking call.
Inputs:
- Buffer -- buffer -- Buffer from which data are to be
transferred.
This finishes the current IWrite call on this GATStreamable. Not
all data in the Buffer MAY have been sent.
Outputs:
- Integer -- nbytes -- Number of bytes successfully written.
Tests if data has been sent from the current IWrite call on
this GATStreamable.
Outputs:
- Integer -- pos -- Position in the buffer right after the last data sent.
Closes this GATStreamable instance.
GATEndpointGATObjectGATMonitorable, GATAdvertiseable
An GATEndpoint represents an end of a byte stream.
Depending on how a GATEndpoint gets created, it can be
listened to or connected to. In both cases, the endpoint returns a
GATPipe. Hence, a GATEndpoint acts in fact as a GATPipe
factory: multiple GATPipes can be created from it by
repeatedly listening for incoming connections. The behaviour is
similar to listening on a BSD socket.
GATEndpoints obtained from the GATAdvertService cannot be listened to.
GATPipes created from endpoints continue to live after the
GATEndpoint instance is destroyed.
Constructs a GATEndpoint corresponding to the passed
GATSerialization and GATContext .
Inputs:
- GATContext -- context -- Used to broker resources
- GATSerialization -- serialEndpoint -- Serialised GATEndpoint .
Constructs a unconnected instance of this class.
Inputs:
- GATContext -- context -- Used to broker resources.
- GATPreferences -- preferences -- User preferences for this instance.
(OPTIONAL)
Destroys this object GATEndpoint. The destructor does not effect
(e.g. destroy) any GATPipes created from this endpoint.
When a GATEndpoint is obtained from an Advert Directory, it can be
used to create a GATPipe connected to the advertising application,
by calling connect on the GATEndpoint instance. This method takes
no parameters. The GATEndpoint can be destroyed at any time
without affecting the obtained GATPipe.
Output:
- A GATPipe connected to the endpoint.
The creator of an GATEndpoint can use the GATEndpoint to create
GATPipes, which represent incoming connections. This is done by
calling Listen on the GATEndpoint instance. The GATEndpoint can
be destroyed at any time without affecting the obtained GATPipe.
Output:
- A GATPipe connected to the endpoint.
The creator of an GATEndpoint can use the GATEndpoint to create
GATPipes, which represent incoming connections. This is done by
calling Listen on the GATEndpoint instance. This call is
asynchroneous, and returns immediately. The GATPipe creation is
performed on the passed GATPipeListener . The
GATEndpoint can be destroyed at any time without affecting the
obtained GATPipe.
Input:
- GATPipeListener -- listener -- GATPipeListener object which handles
incoming connections.
GATPipeListener
This allows of which
this to process GATPipes
produced by listening on GATEndpoints.
Note: Listeners are inherently language dependent. We have
chosen, for this document, to represent them as interfaces, however in
some languages, such as C or Fortran, they MAY need to be represented
as a function and an a3ssociated data structure.
An of a this
receives GATPipes produced by listening on
GATEndpoints through calls to this .
Input:
- GATPipe -- pipe -- The new GATPipe.
GATPipeGATObjectGATMonitorable, GATStreamable
A GATPipe represents a connection to another . It
GATStreamable and the communication methods are
derived from that .
A GATPipe can get created from an GATEndpoint only, by calling its
Listen or Connect methods.
No operations are defined.
Closes the connection, destroys this GATPipe object.
GATFileStreamGATObjectGATMonitorable, GATStreamable
A GATFileStream represents a seekable connection to open
file, the file MAY be either remote or local. It the
the GATStreamable and all communication are as
documented there. The GATFileStream semantics is similar
to a standard Unix file-descriptor. It provides methods to query
the current position in the file and to seek to new positions.
To write data to a GATFileStream it is necessary to construct
a Buffer, and pack it with data. Similarly, in order to read data
a buffer must be created in which the data will be stored. Since
the file MAY be remote, writes and reads MAY either be blocking, or
asynchronous. Asynchronous writes or reads must be completed by
appropriate call. These are present as a result
of the GATStreamable and are as
documented there.
This creates a GATFileStream attached to the physical file at the
specified GATLocation. The file MAY be opened in several modes:
- GATFileStream.GAT_Read -- Open file for reading. The stream
is positioned at the beginning of the file.
- GATFileStream.GAT_Write -- Truncate file to zero length or
create file for writing. The stream is positioned at
the beginning of the file.
- GATFileStream.GAT_Readwrite -- Open for reading and writing.
The stream is positioned at the beginning of the file.
- GATFileStream.GAT_Append -- Open for appending (writing at end
of file). The file is created if it does not exist. The
stream is positioned at the end of the file.
Inputs:
- GATContext -- context -- Used to broker resources.
- GATLocation -- location -- Location of the file to open.
- Integer -- mode -- Mode to open the file (read, write, readwrite,
or append; class constants).
- GATPreferences -- preferences -- User preferences.
(OPTIONAL)
This creates a GATFileStream attached to the physical file at the
specified GATLocation. The file MAY be opened in several modes:
- GATFileStream.GAT_Read -- Open file for reading. The stream
is positioned at the beginning of the file.
- GATFileStream.GAT_Write -- Truncate file to zero length or
create file for writing. The stream is positioned at
the beginning of the file.
- GATFileStream.GAT_Readwrite -- Open for reading and writing.
The stream is positioned at the beginning of the file.
- GATFileStream.GAT_Append -- Open for appending (writing at end
of file). The file is created if it does not exist.
The stream is positioned at the end of the file.
Closed the stream, destroys this GATFileStream object.
This changes the current position in the file. The position can
either be calculated from the current position in the file or
from the beginning or end.
Inputs:
- Integer -- off -- Offset from whence.
- Integer -- whence -- ``Whence'', one of beginning, current or end of file.
Make sure that any underlying file-writing mechanisms have flushed
data to disk.
-- Constant used to indicate read mode
-- Constant used to indicate write mode
-- Constant used to indicate readwrite mode
-- Constant used to indicate append mode
GATFileGATObjectGATMonitorable, GATAdvertiseable
An abstract representation of a physical file.
An of this presents an abstract,
system-independent view of a physical file. User interfaces and
operating systems use system-dependent pathname strings to identify
physical files. GAT, however, uses an operating system independent
pathname string to identify a physical file. A physical file in GAT
is identified by a URI.
An of this GATFile allows for
various high-level to be performed on a
physical file. For example, one can, with a single API call, copy a
physical file from one location to a second location, move a
physical file from one location to a second location, delete a
physical file, and perform various other operations on a physical
file. The utility of this high-level view of a physical file is
multi-fold. The client of an of this
does not have to concern themselves with the details
of reading every single byte of a physical file when all they wish
to do is copy the physical file to a new location. Similarly, a
client does not have to deal with all the various error states that
can occur when moving a physical file (Have all the various bytes
been read correctly? Have all the various bytes been saved
correctly? Did the deletion of the original file proceed
correctly?); the client simply has to call a single API call and
the physical file is moved.
In this use case the client wishes to move a physical file at
a physical location, specified by the GATLocation
location1, to a physical location, specified by the GATLocation
location2. This is done by creating a GATFile
which corresponds to the physical file at the
GATLocation location1, then calling the Move on
the so created GATFile . This is diagramed in the
figure :
Figure:
A sequence diagram for an of the
GATFile.
|
[scale=.7]FileManagementSequence1
|
Constructs a GATFile corresponding to the passed
GATSerialization and GATContext .
Inputs:
- GATContext -- context -- Used to broker resources.
- GATSerialization -- serialFile -- Serialized GATFile .
Constructs a GATFile which corresponds to the
physical file identified by the passed GATLocation and whose
access rights are determined by the passed GATContext.
Inputs:
- GATLocation -- location -- Location which represents the URI corresponding
to the physical file.
- GATContext -- context -- Used to determine the access rights for this GATFile.
- GATPreferences -- preferences -- User preferences for this GATFile.
(OPTIONAL)
Tests this GATFile for equality with the passed GATObject.
If the given GATObject is not a GATFile, then this immediately returns False.
If the given GATObject is a GATFile, then it is deemed equal to this if
a GATLocation
constructed from this GATFile's location and a GATLocation
constructed from the passed GATFile's
GATLocation are equal as determined by the Equals of GATLocation.
Inputs:
- GATObject -- object -- GATObject to test for equality.
Outputs:
- Bool -- equal -- Boolean indicating equality.
Destroys this GATFile .
This copies the physical file represented by this
GATFile , if such a physical
file exists, to a physical file identified by the passed GATLocation. After successful completion of this
this GATFile will not correspond to the new physical file.
Inputs:
- GATLocation -- targetLocation -- Location to which to copy the physical
file corresponding to this GATFile .
This moves the physical file represented by this
GATFile , if such a physical
file exists,to a physical file identified by the passed GATLocation.
After successful completion of this this GATFile will correspond to the new physical file.
Inputs:
- GATLocation -- targetLocation -- Location to which to move the physical
file corresponding to this GATFile .
This deletes the physical file represented by
this GATFile .
Returns a boolean True if the physical file corresponding to this exists and if the physical file corresponding to this is readable .
Outputs:
- Bool -- readable -- Boolean indicating readability.
Returns a Boolean True if the physical file corresponding to this exists and if the physical file
corresponding to this is writable.
Outputs:
- Bool -- writable -- Boolean indicating writability.
Returns an Integer indicating the length in bytes of the physical file corresponding to this ;
then Integer 0 is returned if the corresponding physical file does not exist or if the corresponding physical
file has length 0.
Outputs:
- Integer -- length -- Length of the physical file corresponding to this , in bytes.
Returns the time at which the corresponding physical file was last
written; the Integer 0 is returned if the corresponding physical
file does not exist.
Outputs:
- GATTime -- lastWriteTime -- Time at which this file
was last written.
GATLogicalFileGATObjectGATMonitorable, GATAdvertiseable
An abstract representation of a set of identical physical files.
A GATLogicalFile is an representation of a set of identical
physical files. This abstraction is useful for a number of
reasons, e.g. if one wishes to replicate a physical file which is
at one GATLocation to a second GATLocation. Normally, one takes
all the data at the first GATLocation and replicates it to the
second GATLocation even though the ``network distance,'' between
the first and second GATLocation may be great. A better solution
to this problem is to have a set of identical physical files
distributed at different locations in ``network space.'' If one
then wishes to replicate a physical file from one GATLocation to
a second GATLocation, GAT can then first determine which physical
file is closest in ``network space'' to the second GATLocation,
chose that physical file as the source file, and copy it to the
destination GATLocation. Similarly, the construct of
a GATLogicalFile allows for migrating programs to, while at
a given point in ``network space,'' use the closest physical file
in ``network space' to its physical location.
A logical file has a logical file name. That name space is unique
in the used namespace. The actual implementation of that
namespace is done in a replica catalog, which maps that logical
name to one or more physical names, as described above.
Since GAT may abstract from various implementations and
incarnations of replica catalogs, the used name space is not
necessarily uniq anymore. During the creation of the logical
file, the catalog (and hence the name space) to be used can be
specified, by adding a valid entry for the key
``catalog''.
The site configuration for GAT, and also the user specific
configuration for GAT can specify what catalogs are available, by
mapping a replica catalog service location to a key string.
Constructs a GATLogicalFile corresponding to the passed
GATSerialization and GATContext .
Inputs:
- GATContext -- context -- Used to broker resources.
- GATSerialization -- serialLogicalFile -- Serialized GATLogicalFile .
This constructor creates a GATLogicalFile with passed name. As
initial member, one location for a physical file MAY be given.
The preferences MAY contain a key ``catalog'', whose
value describes a replica catalog namespace to be used for the
logical file name. If not given, a default catalog is used.
Inputs:
- GATContext -- context -- Used to broker resources.
- String -- name -- name in logical name space
- GATLocation -- location -- Location of one initial physical
file in this GATLogicalFile.
(OPTIONAL)
- GATPreferences -- preferences -- User preferences for this .
(OPTIONAL)
Destroys this GATLogicalFile .
Adds the passed GATFile to the set of physical files represented
by this GATLogicalFile .
Inputs:
- GATFile -- file -- File to add to the set of physical files represented by
this GATLogicalFile .
Removes the passed GATFile from the set of physical files
represented by this GATLogicalFile .
Inputs:
- GATFile -- file -- File to remove from the set of
physical files represented by this GATLogicalFile .
Replicates the logical file represented by this to the physical file specified by the
passed GATLocation.
Inputs:
- GATLocation -- location -- Location of the new physical file.
Returns a List of GATFile each of which is a GATFile corresponding
to a physical file represented by this GATLogicalFile .
Inputs:
- List of GATFiles -- files -- GATFile which
correspond to physical files for this GATLogicalFile
.
Event and Monitoring Subsystem
The GAT Event and Monitoring Subsystem covers both interaction with
Grid monitoring services, and application defined events and
actions.
An application MAY state that it can respond to some command (for
example an externally triggered checkpoint), or provide
additional metrics above those provided by a monitoring service.
The application does this by providing a callback for each command,
and for each piece of information it MAY provide. This is done by
creating an object which the GATRequestListener
interface, and invoking the AddRequestListener of
the GATSelf object.
When a process triggers such a request, the GATEngine
invokes the ProcessRequest on the GATRequestListener,
passing it a GATRequest object identifying the request. This
GATRequest object has an associated GATRequestNotifier object which is
used to return the results of the command or the information request.
The application MAY immediately perform the action or determine
the information and invoke the Respond on the
GATRequestNotifier, or it MAY invoke the Respond at a later
time. E.g. in the case of checkpointing the application MAY need to
return to its normal control flow and only later notify the GATEngine
that checkpointing has been done.
Currently the only supported command is checkpointing. The GATTable
passed to the GATRequestNotifier SHOULD contain the following keys
- Checkpoint Files -- a String with paths to any
associated checkpoint files, quoted and comma-separated.
- Restart Files -- a String with paths to any other files
which MAY be used in a restart case, quoted and comma separated. E.g. new
parameter files, etc.
Information requests are used to provide additional metrics to a
monitoring system. When the AddRequestListener method is called, a
GATTable must be provided, which SHOULD define the ``Metric
parameters'', ``Metric measurement type'', ``Metric data type'' and
``Metric unit'', as detailed in the GATMetric class documentation,
for this measurement. That MAY fail if another Metric with the same
name does already exist -- an error is issued in that case.
The GATTable passed to the GATRequestNotifier must contain the
metric value under the ``Metric value'' key, and the time the event
occurred, measured as milliseconds after January 1, 1970, 00:00:00
GMT, stored under ``Event time''.
The application indicates that it can respond to checkpointing
requests; another GAT application checkpoints it by invoking the
Checkpoint on the corresponding GATJob object.
- The application creates an of a class
GATRequestListener, L.
- The application passes L to GATSelf.AddRequestListener, stating that
it is a ``command'' request listener, and that its name is
``checkpoint''. The GATTable can be empty.
- At appropriate points in the flow of control of the application, it
invokes GATContext.ServiceActions.
- When an adaptor receives a checkpoint request, it creates a
GATRequest, R, and an of a class
GATRequestNotifier, N, and invokes the ProcessRequest
on L.
- When L.ProcessRequest is invoked, the application notes that a
checkpoint has been requested, and saves the GATRequestNotifier, N,
for later use, then returns.
- At a later point in processing, after the return of ServiceActions and
checkpointing has been completed, the application invokes N.Respond.
- When N.Respond is invoked, the adaptor informs the process which
invoked the checkpoint of the data returned in the GATTable, and
destroys the GATRequest, R, and the GATRequestNotifier, N.
The application indicates that it can respond to requests for the
number of iterations since application startup; another GAT
application requests this information by using the monitoring
operations.
- The application creates an of a class
GATRequestListener, L.
- The application passes L to GATSelf.AddRequestListener, stating that
it is an ``information'' request listener, and that its name is
``iteration_counter''. The GATTable provides an empty
String for ``Metric parameters'', ``continuous'' for
``Metric measurement type'', ``integer'' for ``Metric data
type'', and ``iteration'' for ``Metric unit''.
- At appropriate points in the flow of control of the application, it
invokes GATContext.ServiceActions.
- When an adaptor receives a request for this information, it creates a
GATRequest, R, and an of a class
GATRequestNotifier, N, and invokes the ProcessRequest
on L.
- When L.ProcessRequest is invoked, the application determines its
iteration and invokes N.Respond with the iteration stored under
``Metric value'', and the current time stored under ``Event time''.
- When N.Respond is invoked, the adaptor informs the process which
requested the information of the data returned in the GATTable, and
destroys the GATRequest, R, and the GATRequestNotifier, N.
GATAction
An which is by the two
``event classes'' - GATRequest and GATMetricEvent.
GATRequestGATObjectGATAction
An of this represents a
request, an action triggered by a process which is
requesting some information or some action to be performed.
- Information -- A GAT Application MAY specify that it
can provide some information, such as the number of iterations that
have been performed, and associate a GATRequestListener to be used
whenever another application requests such information, e.g. by
using a GATMetricEventListener.
- Command -- A GAT Application MAY specify that it can
perform some operation, such as checkpointing, and associate a
GATRequestListener to be used whenever another application requests
this operation, e.g. by invoking the Checkpoint method on a GATJob
object associated with this application. Currently the only such
operation is the checkpoint command, however further commands MAY be
defined in the future.
of this deal with both classes of requests.
A GATRequest instance contains the following information:
- Request name
- Request notifier
The Request name is used to identify the
request definition (e.g. ``checkpoint'' or a metric name).
The GATRequestNotifier
associated with this request.
A GATRequest is passed to the ProcessRequest method
of a GATRequestListener . On receipt, the GAT
application should perform the action or obtain the requested
information and invoke the Respond method on the associated
GATRequestNotifier when it is ready. Only the originating adaptor
SHOULD invoke the destructor of this .
Used by an adaptor to construct an instance of this class.
Inputs:
- GATContext -- context -- GATContext to be associated with this GATRequest.
- String -- name -- Request name.
- GATRequestNotifier -- notifier -- GATRequestNotifier which
will be invoked by the application when the GATRequest is finished.
Destroys this GATRequest object.
Tests this GATRequest for equality with the passed GATObject.
If the given GATObject is not a GATRequest, then this
immediately returns false.
For two GATRequest to be considered as equal
they must have equal GATRequest names, a String, as determined my
the Equals on String.
Inputs:
- GATObject -- object -- The GATObject to test for equality.
Outputs:
- Bool -- equal -- Boolean indicating equality.
Gets the GATRequest name associated with this GATRequest.
Outputs:
- String -- requestName -- Request name.
Gets the GATRequestNotifier associated with this GATRequest.
Outputs:
- GATRequestNotifier -- requestNotifier -- GATRequestNotifier associated with this GATRequest.
GATRequestListener
This allows of which
this to receive GATRequests.
Note: Listeners are inherently language dependent. We have
chosen, for this document, to represent them as interfaces, however in
some languages, such as C or Fortran, they MAY need to be represented
as a function and an a3ssociated data structure.
An of a this receives GATRequests
through calls to this when it is registered to receive such events.
Inputs:
- GATRequest -- request -- Event which triggered this
call.
GATRequestNotifier
When a GATRequest is created, an
of a class GATRequestNotifier is associated with
it. When the request has been satisfied the application should
invoke the Respond method on this . Only the
originating adaptor SHOULD invoke the destructor on this
.
Finalises a GATRequest, passing any response information back to the
invoker.
Inputs:
- GATTable -- data -- Return Data containing any response data.
Note: Listeners are inherently language dependent. We have
chosen, for this document, to represent them as interfaces, however in
some languages, such as C or Fortran, they MAY need to be represented
as a function and an a3ssociated data structure.
GATMonitorable
which can be by any
model elements which are ``monitorable.'' A model
element is monitorable if it is capable of
externalising its .
In this use case the client wishes to register an
of the GATMetricListener to receive GATMetricEvents
from an of the GATMonitorable. In
other words, the client wants the GATMetricListener
to ``monitor'' the GATMonitorable
. The client does so by obtaining an
of the GATMetric from the
GetMetrics on the GATMonitorable
, then providing appropriate values for the various
parameters of this GATMetric . After doing so, the
client passes the so modified GATMetric along
with the desired GATMetricListener to the
AddMetricListener. This is diagrammed in the
figure
Figure:
A sequence diagram for an of the GATMonitorable.
|
[width=]MonitoringSequence
|
Upon successful completion adds the passed of
a GATMetricListener to the list of GATMetricListeners which are
notified of GATMetricEvents by an of
a this . Upon
successful completion of this the passed
GATMetricListener is notified of GATMetricEvents which correspond
to GATMetric passed to this .
The GATMetric passed to this
must be obtained from the GetMetrics on this
, and appropriate values for all the parameters of the
passed GATMetric must be supplied or this
will not complete successfully.
Inputs:
- GATMetricListener -- listener -- GATMetricListener to notify of GATMetricEvents.
- GATMetric -- metric -- GATMetric corresponding to the GATMetricEvents
for which the passed GATMetricListener will be notified.
Upon successful completion removes the passed GATMetricListener
from the list of GATMetricListeners which are notified of
GATMetricEvents corresponding to the passed GATMetric
. The GATMetric passed to this
must be obtained from the GetMetrics
on this , and appropriate values
for all the parameters of the passed GATMetric
must be supplied or this will not complete
successfully.
Inputs:
- GATMetricListener -- listener -- GATMetricListener to notify of GATMetricEvents.
- GATMetric -- metric -- GATMetric corresponding to the GATMetricEvents
for which the passed GATMetricListener has been notified.
Upon successful completion returns a List of GATMetric
. Each GATMetric in this List
corresponds to a metric which can be monitored on this
. To monitor the metric corresponding to
a returned GATMetric one must supply appropriate
values for all the parameters of the GATMetric
and pass the modified GATMetric to the
AddMetricListener with an appropriate
GATMetricListener.
Outputs:
- List of GATMetrics -- metrics -- Each GATMetric
in this List corresponds to a metric which can be monitored on
this .
GATMetricGATObject
An of this represents a
metric, a measurable quantity within a monitoring system.
There are two types of metrics a monitoring system must deal with:
- Local metrics -- Local metrics are metrics that
are directly measured on a resource (e.g. on the local host or on
a running application). These can be highly dependent on the
physical parameters of the resource. Thus, local metrics
originating from two different resources are not necessarily
comparable ( E.g. 1 hour CPU time on a 1 GHz Intel processor is
different than 1 hour CPU time on an 800 MHz PPC processor
although the numeric values are equal.) However resource
administrators who know the configuration of the resource need
local metrics for detailed monitoring of the status and operation
of the resource.
- Grid metrics -- Grid metrics are metrics that
have predefined semantics thus, they are resource independent.
Grid metrics are derived from one or more local metrics by
applying a specific, well defined algorithm (such as unit
conversion, aggregation or averaging). Because of this
transformation grid metrics MAY have less precision or they could
be less specific but are guaranteed to be comparable between
different resources. Unlike local metrics which a local resource
is free to change grid metrics must be agreed upon, standardised
and introduced by community consensus.
of the GATMetric deal with both
types of metrics.
A GATMetric can only be measured if there exists a sensor which
can measure the quantity corresponding to the GATMetric. Such
sensors are created by sensor developers and must embedded in a
resource to allow the corresponding GATMetric or GATMetrics to be
measured. (The creation of sensors is beyond the scope of this
document and as such will not be covered.)
Such sensors, once created, define the GATMetric or GATMetrics
which they allow to be measured.
A metric definition contains the following information:
- Metric name
- Metric parameters
- Metric measurement type
- Metric data type
- Metric Unit
The Metric name is used to identify the metric definition (e.g.
CPU usage). It consists of dot separated words, e.g.
host.cpu.user. The last component of a metric name is the actual
name of the metric, the preceding components are called scope.
The scope can be used to group metrics as well as to
differentiate between similar metrics defined at different levels
(for example, CPU utilisation can be measured on a per-job or
per-host level).
The Metric parameters field in the metric definition contains the
formal definition of the metric parameters. Many metrics can be
measured at different places simultaneously. For example, CPU
utilisation can be measured on several hosts or grid resources.
The metric parameters can be used to distinguish between these
different metric .
The Metric Measurement type can be continuous meaning
data is always available or event-like meaning data only
becomes available when some external event happens (e.g. a
sensor embedded in an application can send events any time).
Continuous metrics are only available using pull model delivery
unless the user specifies a measurement frequency. (The reason
for this is without a specified measurement frequency there is no
event triggering the measurement of a continuous metric.) If the
user asks for periodic measurements by specifying a measurement
frequency the system will generate periodic events automatically.
This avoids polling and allows the system to use the measurement
frequency information to optimise measurements.
The Metric data type contains the definition of the storage used
for representing measurement data.
The Metric unit specifies the physical unit in which the metric
is measured as a String. It is only valid for simple numeric
types and Lists of these types. In the latter case it means the
unit of all elements of the List.
To use the GATMetric one must obtain an
of the GATMetric . One does so
using GetMetrics on a GATMonitorable
. The notion of GATMetric 's
parameters are necessary as the same metric could be measured at
different places (e.g. on different hosts) at the same time. A
GATMetric parameter values are used to
differentiate between these measurements, e.g. instances of a
GATMetric describing the available memory on two host are
distinguished by the values of a parameter containing the
host-name for example.
Tests this GATMetric for equality with the passed GATObject.
If the given GATObject is not a GATMetric, then this immediately returns False.
For two GATMetric to be considered as equal they must have equal GATMetric names, a String, as
determined my the Equals on String. In addition, they must have equal GATMetric parameters
and values as determined by the Equals on GATTable.
Inputs:
- GATObject -- object -- The GATObject to test for equality.
Outputs:
- Bool -- equal -- Boolean indicating equality.
Destroys this GATMetric object.
Gets the GATMetric name associated with this GATMetric.
Outputs:
- String -- metricName -- Metric name.
Gets the GATMetric parameters associated with this GATMetric.
Outputs:
- GATTable -- metricParameters -- Parameters of the GATMetric.
Gets the GATMetric parameter value associated with the passed
GATMetric parameter name. The null Buffer is returned if there is
no GATMetric parameter value with the passed name.
Inputs:
- String -- name -- Parameter name for which to obtain the associated GATMetric parameter value.
Outputs:
- Buffer -- metricParameter -- GATMetric parameter value, a Buffer,
associated with the passed GATMetric parameter name.
Note: Currently, we expect the parameter value returned in
metricParameter to be simple native typed, and hence
castable to simple native types (like integers, floats and
strings). With the future extended handling of typed data in
Buffers, GAT will support more complex values for metrics.
GATMetricEventGATObjectGATAction
An of this represents an metric event, an
event indicating the measurement of a metric and the associated resultant
data.
A metric event occurs whenever a monitored resource, sends out an event to the monitoring system. This
can encompass almost any type of event from disk space running out to memory becoming available. The
various events are defined by the various sensors. This topic is covered in more detail in the GATMetric
documentation.
This returns an of the source of this GATMetricEvent.
Outputs:
- GATObject -- source -- Source of this GATMetricEvent.
This returns the value corresponding to this
GATMetricEvent.
Outputs:
- Buffer -- value -- Contains the value of this GATMetricEvent.
Note: Currently, we expect the parameter value returned in
metricParameter to be simple native typed, and hence
castable to simple native types (like integers, floats and
strings). With the future extended handling of typed data in
Buffers, GAT will support more complex values for metrics.
This returns an instance of the GATMetric to which this GATMetricEvent corresponds.
Outputs:
- GATMetric -- metric -- Corresponds to this GATMetricEvent.
This returns the time when the event happened.
Outputs:
- GATTime -- eventTime -- Time when the event happened.
GATMetricListener
This allows of which
this to receive GATMetricEvents from
which are sources of GATMetricEvents.
Note: Listeners are inherently language dependent. We have
chosen, for this document, to represent them as interfaces, however in
some languages, such as C or Fortran, they MAY need to be represented
as a function and an a3ssociated data structure.
An of a this receives GATMetricEvents
through calls to this when it is registered to receive such events.
Inputs:
- GATMetricEvent -- event -- Event which triggered this call.
Resource Subsystem
This part of the GAT API specification covers classes which are
responsible for interaction with and jobs on the Grid.
A client wishes to schedule a job on any suitable . The
client first creates a GATSoftwareDescription, and fills in all
needed information describing the job and its software environment
(e.g., executable, arguments, environment, required files, etc.).
Then the client creates a GATHardwareResourceDescription, and
fills in all needed information describing the hardware
environment required by the job (e.g., memory, cpu speed, etc).
From these descriptions the client then creates a
GATJobDescription. That GATJobDescription is submitted to a
GATResourceBroker , which returns a GATJob on
successful scheduling.
User creates a GATSoftwareDescription SD.
User creates a GATHardwareResourceDescription HD.
User creates a GATJobDescription using SD and HD.
User creates a GATResourceBroker.
User submits the GATJobDescription to the Resource Broker, and
obtains a GATJob on success.
GATSoftwareDescription SD = new GATSoftwareDescription (...)
GATHardwareResourceDescription HRD = new GATHardwareResourceDescription (...)
GATJobDescription JD = new GATJobDescription (..., SD, HRD)
GATResourceBroker RB = new GATResourceBroker (...)
GATJob J = RB.submit (JD)
delete (SD); delete (HRD);
delete (JD); delete (RB);
This use case is similar to the previous one, but the client picks
a specific compute , and submits the job to that
. To do that, the client again creates a
GATSoftwareDescription and a GATHardwareResourceDescription as
above. The GATHardwareResourceDescription is passed to the
GATResourceBroker , which returns a list of matching
. The client picks one of these and creates
a GATJobDescription from that and the
GATSoftwareDescription. As before, the client submits the
GATJobDescription to the GATResourceBroker, and on
success gets a GATJob returned.
User creates a GATHardwareResourceDescription HRD.
User creates a GATResourceBroker.
User queries the Resource Broker for Resources matching HRD.
User picks one resource from the returned list of resources.
User creates a GATSoftwareDescription SD.
User creates a GATJobDescription using SD and the selected resource.
User submits the GATJobDescription to the Resource Broker, and
obtains a GATJob on success.
GATSoftwareDescription SD = new GATSoftwareDescription (...)
GATHardwareResourceDescription HRD = new GATHardwareResourceDescription (...)
GATResourceBroker RB = new GATResourceBroker (...)
list<GATHardwareResource> HL = RB.findResource (HRD)
GATHardwareResource HR = HL[0]
GATJobDescription JD = new GATJobDescription (..., SD, HR)
GATJob J = RB.submit (JD)
delete (SD); delete (HD);
delete (RB); delete (HL);
delete (HR); delete (JD);
GATSoftwareDescriptionGATObject
An of this is a description of a piece of software
() which is to be submitted as a job. It currently takes
a table describing this piece of software's attributes to any
underlying job submission system.
The GAT-API defines a minimum set of supported name/value pairs to
be included in the GATTable used to construct a
GATSoftwareDescription , as listed in table
. This set MUST be supported by any implementation
of the GAT-API. For any GATSoftwareDescription, the
location MUST be specified.
Table:
Software Description: the minimum set of supported name/values.
| Name |
Type |
Description |
location |
GATLocation |
Software location. |
arguments |
List<String> |
Software arguments. |
environment |
GATTable |
Software environment, names/values are Strings. |
stdin |
GATFile |
Stdin from which the reads. |
stdout |
GATFile |
Stdout to which the writes. |
stderr |
GATFile |
Stderr to which the writes. |
pre-staged files |
List<GATFile> |
Files which SHOULD be staged to the before the is invoked. |
post-staged files |
List<GATFile> |
Files which SHOULD be staged from the after the is finished. |
|
The constructor takes a GATTable describing the
software.
Inputs:
- GATTable -- attributes -- Attributes describing this
software .
Destroys this .
Tests this GATSoftwareDescription for equality with the passed
GATObject. GATSoftwareDescription are equal if they have
equivalent entries in the description table.
Inputs:
- GATObject -- object -- GATObject to test for equality.
Outputs:
- Bool -- equal -- Boolean indicating equality.
GATResourceDescription
The GATResourceDescription interface forms the base for the
GATSoftwareResourceDescriptions and GATHardwareResourceDescriptions
classes; these are used to specify and find resources which may then
be used, for example, to submit a GATJob to. It has an associated
GATTable whose key/value pairs describe the .
A GATJob may have many requirements, both on software and hardware,
which need to be satisfied to run, e.g. specific versions of the
operating system, minimum amount of memory, presence of specific
compilers or libraries on a system, etc. Each one of these
requirements may itself possibly in turn depend on some other
software or hardware requirement. Sometimes there may be possible
alternatives, for example the GATJob may be able to use any one of a
set of possible system libraries which might be installed. Hence, a
complete resource description requires a list of possible
specifications, and, ideally, some way of specifying allowable
alternatives. In order to accommodate this, a
GATResourceDescription has, as well as a table describing software
of hardware resource requirements, a list of child
GATResourceDescriptions, at least one of which must be satisfied in
addition to the requirements of this GATResourceDescription. I.e. a
GATResourceDescription is a tree, and it is matched if a path exists
from the root of the tree to any leaf where the requirements of
every node on that path are met.
Figure:
Classes the GATResourceDescription
contain descriptions of (software or hardware) (a GATTable),
and optionally descriptions of other that
depends on.
|
[width=2/4]GATResourceDescription
|
The user requires that the job be run on hardware which matches
two different GATHardwareResourceDescriptions.
- User creates a GATHardwareResourceDescription HRD1.
- User creates a GATHardwareResourceDescription HRD2.
- User invokes HRD1.AddResourceDescription(HRD2)
Note that this MUST be equivalent to
- User creates a GATHardwareResourceDescription HRD1.
- User creates a GATHardwareResourceDescription HRD2.
- User invokes HRD2.AddResourceDescription(HRD1)
i.e. the order in which requirements appear on any single path
through the tree MUST be irrelevant.
The user requires that the job be run on hardware matching a
certain GATHardwareResourceDescription, but this hardware must have
at least one of LAM or MPICH as MPI implementations.
- User creates a GATHardwareResourceDescription HRD.
- User creates a GATSofwareResourceDescription SRD1,
specifying LAM.
- User invokes HRD.AddResourceDescription(SRD1)
- User creates a GATSofwareResourceDescription SRD2,
specifying MPICH.
- User invokes HRD.AddResourceDescription(SRD2)
The constructor takes an optional GATTable describing the
.
Inputs:
- GATTable -- description -- Description for the
resource.
(OPTIONAL)
Destroys this .
Tests this GATResourceDescription for equality with the passed
GATObject. GATResourceDescriptions are equal if they have
equivalent entries in the description table, and equivalent lists
of dependent GATResourceDescriptions..
Inputs:
- GATObject -- object -- GATObject to test for equality.
Outputs:
- Bool -- equal -- Boolean indicating equality.
This sets the description for this
.
Inputs:
- GATTable -- description -- Description for this
resource's description .
This returns the description for this
.
Outputs:
- GATTable -- description -- Description for the
.
Adds the name/value pair to the GATTable of name/value pairs
which describe the .
Inputs:
- String -- name -- Name to add to the name/value
pairs describing the .
- GATObject -- value -- Value to add to the
name/value pairs describing the .
Removes the name/value pair with the passed name from the
GATTable of name/value pairs which describe the .
Inputs:
- String -- name -- Name to of the name/value pair
to remove from the name/value pairs which describe
the .
This adds an the
GATResourceDescription (GATSoftwareResourceDescription
or GATHardwareResourceDescription) to the list of dependent
GATResourceDescriptions.
Inputs:
- GATResourceDescription -- description --
the GATResourceDescription
to be added to the list.
This removes an the
GATResourceDescription (GATSoftwareResourceDescription
or GATHardwareResourceDescription) from the list of dependent
GATResourceDescriptions. An error is issued if the specified
is not a member of the list of
GATResourceDescriptions.
Inputs:
- GATResourceDescription -- description --
the GATResourceDescription
to be removed from the list.
GATSoftwareResourceDescriptionGATObjectGATResourceDescription
An of this is a description of requisite software
(a ) for some complete resource specification, for example
pre-requisite software for a job to run, or the presence of a
monitoring system to monitor a hardware resource or a running
job. It accepts only specific values for keys on its GATTable
containing the software resource description. These keys are
specified in table .
To clarify the rather vague concept of what a
GATSoftwareResourceDescription describes, the following list gives
some examples:
- a system library,
- a compiler,
- a hosting environment,
- a helper application,
- a component,
- a plugin,
- a service
- an operating system,
E.g., a Fluent job requires the presence of Fluent, a java
application requires a Java run-time environment.
In general any which corresponds to a is
described by a GATSoftwareResourceDescription. However, a hardware
is not described by a GATSoftwareResourceDescription.
The GAT-API defines a minimum set of supported name/value pairs to
be included in the GATTable used to construct a
GATSoftwareResourceDescription , as listed in table
. This set MUST be supported by any implementation
of the GAT-API.
Table:
Software Description: the minimum set of supported name/values.
| Name |
Type |
Description |
os.name |
String |
The os name as returned from uname -s. |
os.type |
String |
The os type as returned from uname -p. |
os.version |
String |
The os version as returned from uname -v. |
os.release |
String |
The os release as returned from uname -r. |
os.name |
String |
The os name as returned from uname -s. |
|
Apart from the specified by the GATResourceDescription
, no other are specified by this class.
GATHardwareResourceDescriptionGATObjectGATResourceDescription
An of this is a description of a hardware
().
To clarify the rather vague concept of what a
GATHardwareResourceDescription describes, the following list gives
some examples:
- compute ,
- network connections,
- a data storage element (e.g.hard drive),
- a graphics hardware resource,
- a supercomputer,
- a laptop,
- or similar pieces of hardware.
In general any which corresponds to a is described
by a GATHardwareResourceDescription. However, software
are not described by a GATHardwareResourceDescription.
The GAT-API defines a minimum set of supported name/value pairs to
be included in the GATTable used to construct a
GATHardwareResourceDescription , as listed in table
. This set MUST be supported by any implementation
of the GAT-API.
Table:
Hardware Description: The minimum set of supported name/values.
| Name |
Type |
Description |
memory.size |
Float |
The minimum memory in GB. |
memory.accesstime |
Float |
The minimum memory access time in ns. |
memory.str |
Float |
The minimum sustained transfer rate in GB/s. |
machine.type |
String |
The machine type as returned from uname -m. |
machine.node |
String |
The machine node name as returned from uname -n. |
cpu.type |
String |
The generic cpu type as returned from uname -p. |
cpu.speed |
Float |
The minimum cpu speed in GHz. |
disk.size |
Float |
The minimum size of the hard drive in GB. |
disk.accesstime |
Float |
The minimum disk access time in ms. |
disk.str |
Float |
The minimum sustained transfer rate in MB/s. |
|
Apart from the specified by the GATResourceDescription
, no other are specified by this class.
GATResource
GATResource is a base which is by any
which wishes to indicate it represents a or ;
currently both a GATHardwareResource and a GATSoftwareResource
this . A GATReservation may be associated with
this GATResource, and can be obtained by the
GetReservation.
Destroys this GATResource , and cancels all pending
reservations associated with it.
Gets the GATResourceDescription which describes this
GATResource .
Outputs:
- GATResourceDescription -- rd -- Describes
this GATResource .
Gets the GATReservation associated with this GATResource
, if any -- otherwise an error is issued.
Outputs:
- GATReservation -- reservation -- An GATReservation
associated with this GATResource.
GATSoftwareResourceGATObjectGATMonitorable, GATResource, GATAdvertiseable
An of this presents an abstract,
system-independent, view of a specific software described
by a GATSoftwareResourceDescription. It allows one to monitor the
, and to examine the various properties of the
to which this corresponds.
Constructs a GATSoftwareResource corresponding to the
passed GATSerialization and GATContext .
Inputs:
- GATContext -- context -- Used to broker .
- GATSerialization -- serialSR -- Serialised
GATSoftwareResource .
Destroys this GATSoftwareResource .
GATHardwareResourceGATObjectGATMonitorable, GATResource, GATAdvertiseable
An of this presents an abstract, system-independent
view of a hardware described by a
GATHardwareResourceDescription . It allows one to monitor the node,
and to examine the various properties of the to which this
corresponds.
Constructs a GATHardwareResource corresponding to the
passed GATSerialization and GATContext .
Inputs:
- GATContext -- context -- Used to broker .
- GATSerialization -- serialHR -- Serialised
GATHardwareResource .
Destroys this GATHardwareResource
GATJobDescriptionGATObject
An of this describes a job to be run. It
consists of a description of the ``executable'' (a
GATSoftwareDescription), and of a description of the
requirements of the job. The latter can be given as either a
GATResourceDescription, or as a specific GATResource; only one of
these may be specified.
Figure:
The class GATJobDescription contains information about
the ``executable'' (GATSoftwareDescription), and of the
the job be executed on (GATResourceDescription or GATResource).
|
[width=3/4]GATJobDescription
|
The GATJobDescription takes a GATContext as argument, and
optionally accepts either one of the following set of arguments:
- (GATSoftwareDescription, GATResourceDescription)
- (GATSoftwareDescription, GATResource)
Inputs:
- GATContext -- context -- GATContext used to broker .
- GATSoftwareDescription -- sd -- Description of the job
executable.
- GATResourceDescription -- rd -- Description of the
the job can be run on.
The second constructor, which accepts a GATResource instead of
a GATResourceDescription.
Inputs:
- GATContext -- context -- GATContext used to broker .
- GATSoftwareDescription -- sd -- Description of the job
executable.
- GATResource -- res -- the job SHOULD be run on.
Destroys this GATJobDescription .
GATJobGATObjectGATMonitorable, GATAdvertiseable
An of this represents a distinct unit which is
submitted to a resource broker.
A GATJob gets created by a GATResourceBroker , on
successful submission of a GATJobDescription.
Upon creation, the GATJob will not, in general, have been submitted
to a specific resource, or queueing system. As time progresses the
GATJob's will be submitted to a jobs scheduler somewhere, and
eventually start to run. Thus a GATJob has various possible :
- GAT_Initial
- GAT_Scheduled
- GAT_SubmissionError
- GAT_Running
- GAT_Stopped
The state may change automatically, or by the some
performed on the GATJob. The states are defined as follows
The GATJob has been submitted to an underlying
management service but has not yet been submitted to a specific
resource.
The GATJob has been submitted by an underlying
management service to a , and the GATJob is scheduled to run.
An error occurred when trying to submit this job, either to an
underlying management service, or when such a service
tried to submit to a queuing system or otherwise dispatch the job.
A description of the error SHOULD be available from a GATStatus
available by invoking the GetStatus .
The represented by the GATJob is
being executed.
GAT_Stopped
The GATJob
was running but is not currently running. That state can be
reached by a successful call to the Stop, or due to
the GATJob completing, or crashing.
Figure:
A state diagram for an of the GATJob.
The State can be actively changed by calling the
unschedule and stop. The bold arrows are state changes
induced by the resource management systems, and are out of
direct user control.
|
[width=1/2]GATJob_States
|
The GATJob does not have a publicly available --
get created by a GATResourceBroker .
The initial GATContext used by the GATJob is that of the creating
GATResourceBroker . The only other way to create
a GATJob is by de-serialisation the class implements
(it the GATAdvertisable ).
Destroys this GATJob object.
Unschedule this GATJob. Upon successful completion this method
guarantees that this GATJob is not scheduled to a job queue, its
state is GAT_Initial. This can only be called on
a GATJob in the GAT_Scheduled state, otherwise an error will is
issued.
Trigger a checkpoint for the GATJob. The call can only succeed
on processes which support application level checkpointing, or on
which provide system level checkpointing. The call returns
immediately after delivering the checkpointing request to the job
or to the resource the job is running on; the actual checkpoint
may happen some time after this return.
During checkpointing, the process state is saved to long term
storage. That state information SHOULD be sufficient to restart
the job, even on a different (migration) or with
multiple copies (cloning). This is useful for which
involve significant data manipulation and can not, for any number
of reasons, finish all of this data manipulation in a single run;
also, it is useful for which involve significant data
manipulation and are, for any number of reasons, unstable. In
both of these cases checkpointing allows the to
manipulate some data now, then, at a later date, continue
running. This can only be called on a GATJob in the
GAT_Running state, otherwise an error will be issued.
The Clone creates a copy of the GATJob. The resulting
GATJob has the same GATSoftwareDescription in its
GATJobDescription, but the GATResourceDescriptions or
GATResources of its GATJobDescription may be altered.
This upon success completes the following steps:
- Constructs a new GATJobDescription with the
GATSoftwareResourceDescription used to construct
this GATJob .
- Complete the new GATJobDescription with the
optional GATHardwareResource passed to the Clone .
- If no GATHardwareResource is given, use the
GATHardwareResourceDescription or GATHardwareResource
used to construct the current job.
- Configures the new GATJobDescription so
that when it begins running it will have the same
as the saved in the last call to
checkpoint on this GATJob .
- Returns this new GATJob to the caller.
This can only be called on a GATJob on which
the Checkpoint has been successfully called at least
once, otherwise an error will be issued.
Inputs:
- GATHardwareResource -- hr -- Hardware resource the
clone SHALL run on.
(Optional)
Outputs:
- GATJob -- clone -- Cloned GATJob .
The Migrate provides similar functionality to the
Clone . The only difference is that the calling GATJob
is discontinued after the new job is spawned off
successfully -- its state is GAT_Stopped then.
Inputs:
- GATHardwareResource -- hr -- Hardware resource the
clone SHALL run on.
(Optional)
Outputs:
- GATJob -- newJob -- Migrated GATJob .
Stops the GATJob. Upon a successful call to this , the
associated with the GATJob are forcibly terminated. This
can only be called on a GATJob in the GAT_Running
state.
This returns the GATJobDescription used to
create that GATJob .
Outputs:
- GATJobDescription -- jobDescription -- GATJobDescription
for this GATJob .
This returns the state of the represented .
This is one of the associated class constants
GAT_Initial, GAT_Scheduled, GAT_Running, or GAT_Stopped.
Outputs:
- Integer -- state -- State of the represented .
This returns an of the GATTable.
Outputs:
- GATTable -- info -- Contains information about the associated GATJob.
This GATTable contains a set of key/value pairs the key, a String,
being the name of the information and the value being the value of
the associated named information. The minimum set of keys which
the returned GATTable contains is as follows:
- hostname
- scheduletime
- starttime
- stoptime
- checkpointable
The key hostname corresponds to a String value which is the name
of the host on which the represented is running, if
GATJob is in the GAT_Running state, or will be running on, if
GATJob is in the GAT_Scheduled state. If the associated GATJob
is not in the GAT_Running or GAT_Scheduled state, then the
value is empty.
The key scheduletime corresponds to an Integer value which is the
number of milliseconds after January 1, 1970, 00:00:00 GMT when
the represented was scheduled. This value is empty for a
GATJob in the GAT_Initial state.
The key starttime corresponds to an Integer value which is the
number of milliseconds after January 1, 1970, 00:00:00 GMT when
the represented was started. This value is empty
for a GATJob in the GAT_Scheduled or GAT_Initial states.
The key stoptime corresponds to an Integer value which is the
number of milliseconds after January 1, 1970, 00:00:00 GMT when
the represented stopped, for a GATJob in the
GAT_Stopped state, otherwise it is empty.
The key checkpointable corresponds to a boolean value. This value
indicates if the represented is able to be
checkpointed.
Other key/value pairs will be in future added to the list of
key/value pairs returned in this GATTable as the need develops.
This returns the job id, a globally unique identifier
for the represented corresponding to this . This
SHOULD be called on a GATJob only when the
is in a GAT_Running or GAT_Scheduled , otherwise
an error will be issued.
Outputs:
- String -- jobID -- Job ID.
When the job is in the GAT_SubmissionError state, this
SHOULD provide a GATStatus which gives further information
as to the cause of the error.
Outputs:
- GATStatus -- status -- An object providing further
information about the submission error.
The following integer constant are used to determine the state of an
of this :
- GAT_Initial
- GAT_Scheduled
- GAT_SubmissionError
- GAT_Running
- GAT_Stopped
GATResourceBrokerGATObject
An of this is used to broker .
A can either be a or a .
Resources are found by constructing a GATResourceDescription
specifying the desired properties of the resource or
resources, e.g. amount of memory, amount of disk-space, installed
software, etc.
Resources may either be found and reserved in one step by calling
the ReserveResource with a GATResourceDescription, and,
optionally, information describing the desired starting time and
duration of the reservation, or resources may first be found by
invoking the FindResources , which returns a list of
GATResources satisfying the GATResourcedescription, and then
the ReserveResource may be invoked with a specific
resource (and optional time information); in either case a
GATReservation is obtained.
A GATResource may then be extracted from the
GATReservation, and this GATResource used to construct a
GATJobDescription.
GATResources found by the FindResources may also be
monitored.
In this use case the client wishes to obtain a reservation for
a hardware for a particular time period.
User creates a GATTimePeriod TP.
User creates a GATHardwareResourceDescription HRD.
User creates a GATResourceBroker.
User submits the GATHardwareResourceDescription and the
GATTimePeriod to the Resource Broker, and
obtains a GATReservation on success.
GATTimePeriod TP = new GATTimePeriod (...)
GATHardwareResourceDescription HRD = new GATHardwareResourceDescription (...)
GATResourceBroker RB = new GATResourceBroker (...)
GATReservation R = RB.ReserveResource (HRD, TP)
delete (TP); delete (HRD);
delete (RB); delete (R);
This constructs a GATResourceBroker
using the passed GATContext and GATPreferences.
Inputs:
- GATContext -- context -- Used to broker .
- GATPreferences -- preferences -- User preferences.
(OPTIONAL)
- GATString -- vo -- Used to indicate the virtual organization
to which this resource broker is to be bound. .
Destroys this GATResourceBroker object.
This attempts to reserve the specified at
the specified time, for the specified time period. Upon success
this returns a GATReservation, which has an associated
GATResource attribute representing the successfully reserved
. Upon failing to reserve the specified this
issues an error.
Inputs:
- GATResourceDescription -- rd -- Description
of the to reserve.
- GATTime -- time -- Time for reservation to start.
(OPTIONAL)
- GATTimePeriod -- duration -- Time period for which
to reserve the .
(OPTIONAL)
Outputs:
- GATReservation -- reservation -- Reservation upon
success.
This attempts to reserve the specified at
the specified time, for the specified time period. Upon success
this returns a GATReservation, which has an associated
GATResource attribute representing the successfully reserved
. Upon failing to reserve the specified this
issues an error.
Inputs:
- GATResource -- res -- GATResource
corresponding to the to reserve.
- GATTime -- time -- Time for reservation to start.
(OPTIONAL)
- GATTimePeriod -- duration -- Time period for which
to reserve the .
(OPTIONAL)
Outputs:
- GATReservation -- reservation -- Reservation upon
success.
This attempts to find one or more matching
(s). Upon success this returns a List of
GATResource . Upon failing to find any specified
an empty list is returned.
Inputs:
- GATResourceDescription -- rd -- Description of the
(s) to find.
Outputs:
- List of GATResources -- resources -- found.
This takes a GATJobDescription, and submits the
specified job to some underlying resource management or allocation
system. On success, a GATJob is returned, which
represents the job. Upon failing to submit the job, an
error is issued.
Inputs:
- GATJobDescription -- jobDescription -- Description of the job to
schedule.
Outputs:
- GATJob -- job -- Representation of the job.
GATReservationGATObject
An of this is a reservation for a GATResource.
This upon successful completion cancels the
reservation corresponding to this GATReservation
.
This returns the GATResource corresponding to this
GATReservation . That can in turn call the
GetReservation to obtain this GATReservation
.
Outputs:
- GATResource -- resource -- upon success,
undefined otherwise.
Next: 4 GAT Application Utility
Up: 7 <
Previous: 2 API Descriptors List
Contents
Andre Merzky
2004-05-20
| 
