This example job illustrates how to schedule a "sweep," which is an ordered, serialized scheduling of the joblets across all matching resources.
> zos login --user zenuser
Please enter current password for 'zenuser':
Logged into grid as zenuser
> zos jobinfo --detail sweeper
Jobname/Parameters Attributes
------------------ ----------
sweeper Desc: This example job ilustrates how to schedule a 'sweep'
accross all matching resources.
sleeptime Desc: time to sleep (in seconds)
Type: Integer
Default: 1
A representation of a running job instance.
Defines execution on the resource.
A representation of the matrix grid object, which provides operations for retrieving and creating grid objects in the system. MatrixInfo is retrieved using the built-in getMatrix() function. Write capability is dependent on the context in which getMatrix() is called. For example, in a joblet process on a resource, creating new grid objects is not supported.
A representation of Group grid objects. Operations include retrieving the group member lists and adding/removing from the group member lists, and retrieving and setting facts on the group.
Used to manage command line execution on resources.
Specifies the time in seconds that the job remains dormant before running (default 1).
The files that make up the Sweeper job include:
sweeper # Total: 140 lines |-- sweeper.jdl # 66 lines `-- sweeper.policy # 74 lines
The ScheduleSpec utility class is also related to this example.
1 # -----------------------------------------------------------------------------
2 # Copyright © 2010 Novell, Inc. All Rights Reserved.
3 #
4 # NOVELL PROVIDES THE SOFTWARE "AS IS," WITHOUT ANY EXPRESS OR IMPLIED
5 # WARRANTY, INCLUDING WITHOUT THE IMPLIED WARRANTIES OF MERCHANTABILITY,
6 # FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGMENT. NOVELL, THE AUTHORS
7 # OF THE SOFTWARE, AND THE OWNERS OF COPYRIGHT IN THE SOFTWARE ARE NOT LIABLE
8 # FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
9 # TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE
10 # OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
11 # -----------------------------------------------------------------------------
12 # $Id: sweeper.jdl 10344 2009-11-20 21:46:43Z jastin $
13 # -----------------------------------------------------------------------------
14
15 import time
16
17 #
18 # Add to the 'examples' group on deployment
19 #
20 if __mode__ == "deploy":
21 try:
22 jobgroupname = "examples"
23 jobgroup = getMatrix().getGroup(TYPE_JOB, jobgroupname)
24 if jobgroup == None:
25 jobgroup = getMatrix().createGroup(TYPE_JOB, jobgroupname)
26 jobgroup.addMember(__jobname__)
27 except:
28 exc_type, exc_value, exc_traceback = sys.exc_info()
29 print "Error adding %s to %s group: %s %s" % (__jobname__, jobgroupname, exc_type, exc_value)
30
31
32 class sweeperJob(Job):
33
34 def job_started_event(self):
35 self.setFact("jobinstance.memo", self.getFact("job.description"))
36
37 sp = ScheduleSpec()
38
39 # Optionally a constraint can be specified to further limit matching
40 # resources from the job's default 'resource' constraint. Could also
41 # compose an object Constraint.
42 # For example, uncomment to restrict to resource group 'sweeper'
43 #sp.setConstraint("<contains fact='resource.groups' value='sweeper' />")
44
45 # Specify the joblet to run on each resource
46 sp.setJobletClass(sweeperJoblet)
47
48 # Specify the sweep across active nodes
49 sp.setUseNodeSet(sp.ACTIVE_NODE_SET)
50
51 # Schedule a sweep (creates preassigned joblets)
52 self.scheduleSweep(sp)
53
54 # Now the ScheduleSpec contains the number of joblets created
55 print 'Launched', sp.getCount(), 'joblets'
56
57
58 class sweeperJoblet(Joblet):
59
60 def joblet_started_event(self):
61 msg = "run on resource %s" % (self.getFact("resource.id"))
62 self.setFact("joblet.memo", msg)
63 print "Sweep", msg
64 sleeptime = self.getFact("jobargs.sleeptime")
65 time.sleep(sleeptime)
66
1 <!-- 2 *============================================================================= 3 * Copyright (c) 2010 Novell, Inc. All Rights Reserved. 4 * 5 * NOVELL PROVIDES THE SOFTWARE "AS IS," WITHOUT ANY EXPRESS OR IMPLIED 6 * WARRANTY, INCLUDING WITHOUT THE IMPLIED WARRANTIES OF MERCHANTABILITY, 7 * FITNESS FOR A PARTICULAR PURPOSE, AND NON INFRINGMENT. NOVELL, THE AUTHORS 8 * OF THE SOFTWARE, AND THE OWNERS OF COPYRIGHT IN THE SOFTWARE ARE NOT LIABLE 9 * FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 10 * TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE 11 * OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 12 *============================================================================= 13 * $Id: sweeper.policy 10344 2009-11-20 21:46:43Z jastin $ *============================================================================= 15 --> 16 17 <policy> 18 19 <jobargs> 20 <!-- 21 - Defines and sets the length of time the joblet should pretend 22 - it is doing something important 23 --> 24 <fact name="sleeptime" 25 type="Integer" 26 description="time to sleep (in seconds)" 27 value="1" 28 visible="true" /> 29 </jobargs> 30 31 32 <job> 33 <!-- 34 - Give the job a description for GUI's 35 --> 36 <fact name="description" 37 type="String" 38 value="This example job ilustrates how to schedule a 'sweep' accross all matching resources." /> 39 40 <!-- 41 - This activates a built in throttle to limit the number of 42 - resources this job will run on at a time 43 --> 44 <fact name="maxresources" 45 type="Integer" 46 value="3" /> 47 48 <!-- 49 - Rank resources from least loaded to the highest loaded. The 50 - idea is to run the joblets on the least loaded node first 51 - and hopefully by the time we get to the higher loaded machines 52 - their load may have gone down 53 --> 54 <!-- 55 <fact name="resources.rankby"> 56 <array> 57 <string>resource.loadaverage/a</string> 58 </array> 59 </fact> 60 --> 61 62 <!-- 63 - Alternative ranking that is easier to see: 64 - decending alphabetic of node name 65 --> 66 <fact name="resources.rankby"> 67 <array> 68 <string>resource.id/d</string> 69 </array> 70 </fact> 71 </job> 72 73 </policy> 74
The class sweeperJob (see line 32, sweeper.jdl) is derived from the Job Class.
The class sweeperJoblet (see line 58, sweeper.jdl) is derived from the Joblet Class.
The sweeper.job can be broken down into four separate parts:
In addition to specifying the jobarg and default settings for sleeptime in lines 24-28, sweeper.policy), there also is the <job/> section in lines 32-71, which describes static facts..
The resources.rankby array has two notable setting in this example:
resource.loadaverage: This is the first string assignment (lines 55-59), which is commented out, that causes joblets to run on the least loaded nodes first.This is the default value and the default launch order for scheduleSweep.
resource.id: This is the second string assignment (lines 66-70), which is actually used, and assigns the string to the rank by array so that joblets run on nodes in reverse alphabetical order.
When the Orchestrate Server deploys a job for the first time (see Section 3.5, Deploying Jobs), the job JDL files are executed in a special deploy mode. When sweeper.jdl is run in this way (either through the Development Client or the zosadmin deploy command), lines 20-29 are executed. This attempts to locate the examples jobgroup (lines 22-23), creates the group if it is not found (lines 24-25), and adds the sweeper job to the group (line 26).
If the deployment fails for any reason, then an exception is thrown (line 27), which prints the job name, group name, exception type and value (line 29).
The sweeperJob class (line 32) defines only the required job_started_event (line 34) method. This method runs on the Orchestrate Server when the job is run to launch the joblets.
When executed, job_started_event displays a message on the memo line of the Job Log tab within the Jobs view in the Orchestrate Development Client (line 35), via jobinstance.memo (see Section 3.10.1, Creating a Job Memo).
Jumping ahead for a moment, instead of calling self.schedule() as most the other examples do to instantiate joblets, sweeperJob calls self.scheduleSweep() (line 52). scheduleSweep requires an instance of ScheduleSpec, so one is created (line 37).
The ScheduleSpec method setConstraint can be used to constrain the available resources to a particular group, as shown with a comment (line 43). If this setConstraint line is uncommented, joblets will only run on members of the sweeper resource.group instead of using the default resource group all.
NOTE:The sweeper group must already be created and have computing nodes assigned to it (see Walkthrough: Creating a Resource Account
in the PlateSpin Orchestrate 2.5 Installation and Configuration Guide). This constraint would also be ANDed to any existing constraint, including any aggregated policies.
The sweeperJoblet is set to be scheduled (line 52), and setUseNodeSet(intnodeSet) is assigned (line 49) the value sp.ACTIVE_NODE_SET. So, the joblet set is constructed after applying resource constraints to the active/online resources. This in contrast to the other possible value of sp.PROVISIONABLE_NODE_SET, where constraints are applied to all provisionable resources.
The sweeperJoblet class (lines 58-65) defines only the required joblet_started_event (line 60) method. After this method is executed, it displays a message on the memo line of the Joblet tab within the Jobs view in the Orchestrate Development Client (lines 61-62). It also prints a similar log message (line 63), and then just sleeps for jobargs.sleeptime seconds (lines 64-65) before completion.
Execute the following commands to deploy and run sweeper.job:
Deploy notepad.job into the grid:
> zosadmin deploy sweeper.job
Display the list of deployed jobs:
> zos joblist
should appear in this list.
Run the job on one or more resources using the default values for numJoblets and resource, specified in the sweeper.policy file:
> zos run sweeper sleeptime=30 JobID: zenuser.sweeper.420 > zos status zenuser.sweeper.420 Completed > zos log zenuser.sweeper.420 Launched 3 joblets [melt] Sweep run on resource melt [freeze] Sweep run on resource freeze [skate] Sweep run on resource skate
Setting Constraints Using Policies, see Section 2.3, Policies and Section 4.0, Understanding Grid Object Facts and Computed Facts.