com.novell.zos.jdl
Class DataGrid

java.lang.Object
  com.novell.zos.jdl.DataGrid

public class DataGrid
extends java.lang.Object

General interface to the Data Grid.

Interactions with the Data Grid revolve around instances of this class. Operations include copying files from the DataGrid down to the resource for Joblet usage and likewise uploading files from a resource to the DataGrid.

A Data Grid URL is the convention for denoting paths in the Data Grid. The Data Grid URL convention is the form grid:///. The gridID is optional and if absent means the default grid. The ^ symbol can be used as a shortcut to the directory either standalone indicating the current job or followed by the jobid to identify a particular job. The ! symbol can be used as a shortcut to the deployed job's home directory either standalone indicating the current jobs type or followed by the deployed jobs name. The ~ symbol can be used as a shortcut to the users home directory (in the datagrid) either standalone indicating the current user or followed by the desired user id to identify a particular user.

Example to copy a file from the a Datagrid directory to a local file. This example assumes you have created the directory 'newdir' in the DataGrid and have uploaded a file named 'results.txt' there:

         DataGrid().copy("grid:///newdir/results.txt","local.txt")

Example to copy a file from the job instance directory to a local file:

         DataGrid().copy("grid:///^/results.txt","local.txt")

Example to copy a file from a named job instance directory to a local file:

         DataGrid().copy("grid:///^user.myjob.1024/results.txt","local.txt")

Example to copy a local file into the job directory for job 'myjob':

         DataGrid().copy("local.txt","grid:///!myjob")

Example to copy a local file into a subdir of the job directory:

         DataGrid().copy("local.txt","grid:///!myjob/subdir")

Example to copy a local file into the current job instance directory:

         DataGrid().copy("local.txt","grid:///^/")

The DataGrid class is only allowed during Joblet execution on a resource.

Constructor Summary

DataGrid()
Construct a DataGrid instance

Method Summary

void	append(java.lang.String destfile, java.lang.String value) Append the string data to the end of the grid file referenced by this path.
void	copy(java.lang.String source, java.lang.String dest) Fetch the specified source file to the destination.
void	copy(java.lang.String source, java.lang.String dest, java.lang.String[] nonSparse) Copy recursively with a list of non-sparse suffixes.
void	delete(java.lang.String path) Delete the file or directory referenced by this path.
boolean	exists(java.lang.String gridPath) Check if the supplied gridPath exists.
boolean	getCache() Retrieve setting of using cached data or not
boolean	getDebug() Retrieve value of debug logging.
boolean	getMulticast() Return true if operation is to attempt multicast.
boolean	getMulticastFallBack() If true then fall back to unicast if multicast does not mee the setMulticastMin requirement.
int	getMulticastMin() Retrieve the minimum number of receivers needed in order to multicast.
long	getMulticastMinFileSize() Retreive the multicast minimum file size
int	getMulticastQuorum() Retrieve number of receivers that constitute a quorum and allow multicast to start.
int	getMulticastRate() Retrieve the requested data rate in bytes per second for multicast.
int	getMulticastWait() Retrieve maximum amount of time a receiver is prepared to wait before start of multicast (ms).
boolean	getSparse() If true, local files will be written sparsely.
void	mkdir(java.lang.String dir) Recursively create a new empty directory.
void	rename(java.lang.String source, java.lang.String dest) Move a file or directory from one path to another.
void	rmdir(java.lang.String dir) Delete the file or directory referenced by this path.
void	setCache(boolean caching) To use cached data or not.
void	setMulticast(boolean mcast) Attempt to fetch a file as part of a combined multicast transfer.
void	setMulticastFallBack(boolean mcastFallBack) If true, then a multicast attempt that does not meet the setMulticastMin requirement will fall back to a unicast file copy.
void	setMulticastMin(int mcastMin) Set the minimum number of receivers required in order to proceed with the multicast.
void	setMulticastMinFileSize(long minFileSize) Set the multicast minimum file size.
void	setMulticastQuorum(int mcastQuorum) Sets the number of receivers that constitute a quorum and allow multicast to start.
void	setMulticastRate(int mcastRate) Set the multicast send rate limit in bytes per second.
void	setMulticastWait(int mcastWait) Set the number of milliseconds to wait for a more receivers to join the multicast.
void	setRecursive(boolean recursive) Perform recursive copy.
void	setSparse(boolean sparse) To create local files sparsely or not.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataGrid

public DataGrid()

Construct a DataGrid instance

Method Detail

getDebug

public boolean getDebug()

Retrieve value of debug logging.

Returns:

true if debug logging is on, false if not

setCache

public void setCache(boolean caching)

To use cached data or not.

Parameters:

caching - true to use caching, false to not.

getCache

public boolean getCache()

Retrieve setting of using cached data or not

Returns:

true if caching, false if not

setSparse

public void setSparse(boolean sparse)

To create local files sparsely or not.

Parameters:

sparse - true to save sparse files, otherwise false.

getSparse

public boolean getSparse()

If true, local files will be written sparsely.

Returns:

If true, local files will be written sparsely.

setMulticast

public void setMulticast(boolean mcast)

Attempt to fetch a file as part of a combined multicast transfer. This only applies to the copy() operation.

Behaves the same as a unicast copy(), except that an initial attempt is made to fetch the file as part of a multicast to this and potentially many other hosts at the same time. If successful, the total network bandwith consumed by all hosts can be greatly reduced.

If multicasting is not available or not enough hosts are willing to multicast the same file in the specified wait period, then this operation transparently falls back to a normal unicast copy() request.

Multicasting is intended for use by a set of nodes running the same job and requesting the same large file from the data grid. The multicastMin, multicastQuorum, and multicatWait are ignored for requests after the first to join any given multicast session.

Multicast also honors caching requests if caching is used.

Parameters:

mcast - true to attempt multicast. Default is false.

getMulticast

public boolean getMulticast()

Return true if operation is to attempt multicast.

Returns:

true for multicast, false if not.

setMulticastFallBack

public void setMulticastFallBack(boolean mcastFallBack)

If true, then a multicast attempt that does not meet the setMulticastMin requirement will fall back to a unicast file copy. If false, then the copy will fail.

Parameters:

mcastFallBack - true to fallback to unicast, false to not

getMulticastFallBack

public boolean getMulticastFallBack()

If true then fall back to unicast if multicast does not mee the setMulticastMin requirement.

Returns:

true to fallback to unicast, false to not

setMulticastMin

public void setMulticastMin(int mcastMin)

Set the minimum number of receivers required in order to proceed with the multicast. If this number is not reached within the "setMulticastWait" time limit, then the receivers will either fall back to unicast or fail, depending on the "setMulticastFallback" option.

Parameters:

mcastMin - Minimum number of receivers

getMulticastMin

public int getMulticastMin()

Retrieve the minimum number of receivers needed in order to multicast. If not reached, then unicast is used.

Returns:

Minimum number of receivers

setMulticastWait

public void setMulticastWait(int mcastWait)

Set the number of milliseconds to wait for a more receivers to join the multicast. The multicast will begin if this time expires, or if the "setMulticastQuorum" number of receivers is reached. This allows other hosts to join in the requested multicast. Zero wait means the default wait of 15 seconds. -1 means to wait infinitely.

Parameters:

mcastWait - Wait time in milliseconds

getMulticastWait

public int getMulticastWait()

Retrieve maximum amount of time a receiver is prepared to wait before start of multicast (ms).

Returns:

Wait time in milliseconds

setMulticastQuorum

public void setMulticastQuorum(int mcastQuorum)

Sets the number of receivers that constitute a quorum and allow multicast to start.

Once this many receivers join a multicast, the server is free to start sending immediately. A negative or zero value means there is no quorum. The wait time will be allowed to fully expire before sending begins.

Parameters:

mcastQuorum - Number of receivers

getMulticastQuorum

public int getMulticastQuorum()

Retrieve number of receivers that constitute a quorum and allow multicast to start.

Returns:

Number of receivers

setMulticastRate

public void setMulticastRate(int mcastRate)

Set the multicast send rate limit in bytes per second. If zero, then the default rate limit defined on the server is used. This value may be capped by a maximum rate limit on the server, as well.

Parameters:

mcastRate - The requested data rate in bytes per second

getMulticastRate

public int getMulticastRate()

Retrieve the requested data rate in bytes per second for multicast.

Returns:

The requested data rate in bytes per second.

setMulticastMinFileSize

public void setMulticastMinFileSize(long minFileSize)

Set the multicast minimum file size. If file size is equal to or larger than this limit then it is considered for multcast. If it is smaller, then it is always unicast, despite the setting of setMulticast()

Parameters:

minFileSize - Filesize in bytes. Default is 10000000 (10M)

getMulticastMinFileSize

public long getMulticastMinFileSize()

Retreive the multicast minimum file size

Returns:

Minimum file size for multicasting

setRecursive

public void setRecursive(boolean recursive)

Perform recursive copy. Default is to not recurse.

Parameters:

recursive - True for the copy() to do a recursive copy, including creating any directories.

copy

public void copy(java.lang.String source,
                 java.lang.String dest)
          throws java.lang.Exception

Fetch the specified source file to the destination. A recursive copy will be attempted if setRecursive(True) is set. The default is a single file copy. A multicast will be attempted if setMulticast(True) is set. The default is to do a unicast copy.

Example to copy down a file from the DataGrid to a resource and then read the lines of the file:

      datagrid = DataGrid()
      datagrid.copy("grid:///images/myFile","myLocalFile")
      text = open("myLocalFile").readlines()
 

Example to recursively copy a directory and it's sub directories from the DataGrid to a resource.

      datagrid = DataGrid()
      datagrid.setRecursive(True)
      datagrid.copy("grid:///testStore/testFiles","/home/tester/myLocalFiles")
 

Example to copy down a file from the Job deployment area to a resource and then read the lines of the file:

      datagrid = DataGrid()
      datagrid.copy("grid:///!myJob/myFile","myLocalFile")
      text = open("myLocalFile").readlines()
 

Either the source or the destination must be a DataGrid URL. A local copy operation is not supported.

Parameters:

source - Full grid: path or local file path of source

dest - Full grid: path or local file path of destination

Throws:

java.lang.Exception - If any error occurs in copy

java.lang.Exception - Thrown if a system or configuration error prevents the copy of files to the grid or network failure or I/O error prevents the copy of files to the grid.

copy

public void copy(java.lang.String source,
                 java.lang.String dest,
                 java.lang.String[] nonSparse)
          throws java.lang.Exception

Copy recursively with a list of non-sparse suffixes. Used to allow vdisk copy to copy sparseness in-line for efficiency.

Parameters:

source - Full grid: path or local file path of source

dest - Full grid: path or local file path of destination

nonSparse - List of file suffixes to not be sparse. Compared using endsWith()

Throws:

java.lang.Exception - If any error occurs in copy

java.lang.Exception - Thrown if a system or configuration error prevents the copy of files to the grid or network failure or I/O error prevents the copy of files to the grid.

mkdir

public void mkdir(java.lang.String dir)
           throws java.lang.Exception

Recursively create a new empty directory.

If no component in the path referenced by this object is a regular file, and the current user has permission to do so, then a directory is created that reflects the current data grid path. Any intermediate subdirectories are automatically created as well.

Parameters:

dir - Directory path to create

Throws:

java.lang.Exception

rmdir

public void rmdir(java.lang.String dir)
           throws java.lang.Exception

Delete the file or directory referenced by this path.

If this path references a file, then the file is removed from the grid. If a directory is referenced, then the directory is recursively deleted.

Parameters:

dir - Directory to delete

Throws:

java.lang.Exception

delete

public void delete(java.lang.String path)
            throws java.lang.Exception

Delete the file or directory referenced by this path.

If this path references a file, then the file is removed from the grid. If a directory is referenced, then the directory is recursively deleted.

Parameters:

path - File or directory to delete

Throws:

java.lang.Exception

rename

public void rename(java.lang.String source,
                   java.lang.String dest)
            throws java.lang.Exception

Move a file or directory from one path to another.

Parameters:

source - Source to move

dest - Destination of move

Throws:

java.lang.Exception

append

public void append(java.lang.String destfile,
                   java.lang.String value)
            throws java.lang.Exception

Append the string data to the end of the grid file referenced by this path.

If the file does not exist, it is created.

Parameters:

destfile - File to append to

value - String data to append

Throws:

java.lang.Exception - Thrown if a system or configuration error prevents the append of data to the grid file.

exists

public boolean exists(java.lang.String gridPath)
               throws java.lang.Exception

Check if the supplied gridPath exists.

Example to check if an executable is included as part of the "myJob" deployment. If the file exists, then the file is copied down to the resource.

      if DataGrid().exists("grid:///jobs/myJob/myUtility.exe"):
           DataGrid().copy("grid:///jobs/myJob/myUtility.exe","myUtility.exe")

Parameters:

gridPath - The Data Grid file URL string to use as the path. (required)

Returns:

true if gridPath exists, false if does not

Throws:

java.lang.Exception - if errors occurs checking for existence or gridPath is invalid