© 2012 ibm corporation using shadow jobs to manage cross-platform dependencies

© 2012 IBM Corporation

Using Shadow Jobs to Manage Cross-Platform Dependencies

© 2012 IBM Corporation2

Workload Automation

Agenda

• Introduction• Cross dependencies flow• Binding process• Shadow job status changes• Setup• Demo / Q&A

© 2012 IBM Corporation3

Workload Automation

Introduction (I)• Workload is sometimes spread across different scheduling

environments, often located in different sites Scheduling environments managed independently Scheduling environments need to interoperate

• Using cross dependencies Synchronize activities between different scheduling environments

© 2012 IBM Corporation4

Workload Automation

Introduction (II)• Job B (Local Engine) logically dependent on Job A (Remote Engine)• Job A’ (on Local Engine) represents Job A (on Remote Engine)• Local Engine

Tivoli Workload Scheduler Distributed Tivoli Workload Scheduler for z/OS (Session 3603)

• Remote Engine Tivoli Workload Scheduler Distributed Tivoli Workload Scheduler for z/OS

© 2012 IBM Corporation5

Workload Automation

Cross dependencies flow

Shadow Job on the local engine is free from dependencies

© 2012 IBM Corporation6

Workload Automation

Cross dependencies flow

Broker of the local engine sends binding request to the remote engine

© 2012 IBM Corporation7

Workload Automation

Cross dependencies flow

Broker of the remote engine sends binding response to the local engine Broker of the local engine updates the status of the Shadow Job

© 2012 IBM Corporation8

Workload Automation

Cross dependencies flow

Remote Job starts running Broker of the remote engine sends the status update to the local engine Broker of the local engine updates the status of the Shadow Job

© 2012 IBM Corporation9

Workload Automation

Cross dependencies flow

Remote Job completes successfully Broker of the remote engine sends the status update to the local engine Broker of the local engine updates the status of the Shadow Job Successor of the Shadow Job is released

© 2012 IBM Corporation10

Workload Automation

Binding request (remote engine = TWSd)<jmgr:JobDefinitionDocument>

<jsdl:jobDefinition>

<jsdl:application name="distributedShadowJob">

<dshadow:DistributedShadowJob>

<dshadow:Timezone>Europe/Brussels</dshadow:Timezone>

<dshadow:StartOfDay>2300</dshadow:StartOfDay>

<dshadow:JobStream>JS1</dshadow:JobStream> <dshadow:Workstation>WS1</dshadow:Workstation> <dshadow:Job>XYZ</dshadow:Job> <dshadow:matching> <dshadow:sameDay/> </dshadow:matching> <dshadow:ScheduledTime>2012-04-12T13:00:00Z</dshadow:ScheduledTime> <dshadow:SchedDate>2012-04-12T00:00:00Z</dshadow:SchedDate>

</dshadow:DistributedShadowJob>

</jsdl:application>

<jsdl:resources/>

</jsdl:jobDefinition>

</jmgr:JobDefinitionDocument>

<jmgr:Alias>TWS3_REM_ENG#JOBS.TEST_L1636311662.JNUM-560798319</jmgr:Alias>

<jmgr:JobId>2f39b341-0ec6-311c-a9b7-6f27e33a2229</jmgr:JobId>

<jmgr:ClientNotifyURI>https://tws1.swg.be.ibm.com:31116/JobManagerRESTWeb/JobScheduler/job</jmgr:ClientNotifyURI> <jmgr:ClientNotifyURI>https://tws2.swg.be.ibm.com:31116/JobManagerRESTWeb/JobScheduler/job</jmgr:ClientNotifyURI>

© 2012 IBM Corporation11

Workload Automation

Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”

© 2012 IBM Corporation12

Workload Automation

Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”

If a matching job stream instance in found in the Preproduction Plan

© 2012 IBM Corporation13

Workload Automation

Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”

If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”

© 2012 IBM Corporation14

Workload Automation

Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”

If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”

Look for the job in matching job stream instance in Current Plan

(Example : XYZ )

© 2012 IBM Corporation15

Workload Automation

Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”

If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”

Look for the job in matching job stream instance in Current Plan

(Example : XYZ ) If found

Send INFO notification back to the local engine (Shadow Job = BOUND).

© 2012 IBM Corporation16

Workload Automation

Binding process (remote engine = TWSd)

© 2012 IBM Corporation17

Workload Automation

Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”

If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”

Look for the job in matching job stream instance in Current Plan

(Example : XYZ ) If found

Send INFO notification back to the local engine (Shadow Job = BOUND). Else

Send ERROR notification back to the local engine (Shadow Job = ERROR).

© 2012 IBM Corporation18

Workload Automation

Binding process (remote engine = TWSd)

© 2012 IBM Corporation19

Workload Automation

Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”

If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”

Look for the job in matching job stream instance in Current Plan

(Example : XYZ ) If found

Send INFO notification back to the local engine (Shadow Job = BOUND). Else

Send ERROR notification back to the local engine (Shadow Job = ERROR). Else

Send INFO notification back to the local engine (Shadow Job = BOUND).

© 2012 IBM Corporation20

Workload Automation

Binding process (remote engine = TWSd)

© 2012 IBM Corporation21

Workload Automation

Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”

If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”

Look for the job in matching job stream instance in Current Plan

(Example : XYZ ) If found

Send INFO notification back to the local engine (Shadow Job = BOUND). Else

Send ERROR notification back to the local engine (Shadow Job = ERROR). Else

Send INFO notification back to the local engine (Shadow Job = BOUND). Else

Send ERROR notification back to the local engine (Shadow Job = ERROR).

© 2012 IBM Corporation22

Workload Automation

Binding process (remote engine = TWSd)

© 2012 IBM Corporation23

Workload Automation

Binding process (remote engine = TWSd)• If “Scheduled time of Shadow Job” ≤ “Preproduction Plan end time”

If a matching job stream instance in found in the Preproduction Plan If “Scheduled time of Shadow Job” ≤ “Current Plan end time”

Look for the job in matching job stream instance in Current Plan

(Example : XYZ ) If found

Send INFO notification back to the local engine (Shadow Job = BOUND). Else

Send ERROR notification back to the local engine (Shadow Job = ERROR). Else

Send INFO notification back to the local engine (Shadow Job = BOUND). Else

Send ERROR notification back to the local engine (Shadow Job = ERROR).

• Else Delay notification to the local engine (Shadow Job remains in WAIT ).

© 2012 IBM Corporation24

Workload Automation

Binding request (remote engine = TWSz)<jmgr:JobDefinitionDocument>

<jsdl:jobDefinition>

<jsdl:application name=“zShadowJob">

<zshadow:zShadowJob>

<zshadow:Timezone>Europe/Brussels</zshadow:Timezone>

<zshadow:JobStream>APP1</zshadow:JobStream> <zshadow:JobNumber>002</zshadow:JobNumber>

<zshadow:matching> <zshadow:previous/> </zshadow:matching> <zshadow:ScheduledTime>2012-04-15T15:00:00Z</zshadow:ScheduledTime>

</zshadow:zShadowJob>

</jsdl:application>

<jsdl:resources/>

</jsdl:jobDefinition>

</jmgr:JobDefinitionDocument>

<jmgr:Alias>TWS4_REM_ENG#JOBS.TEST_L1636311663.JNUM-560799234</jmgr:Alias>

<jmgr:JobId>2f39b341-0ec6-311c-a9b7-6f27e34r2229</jmgr:JobId>

<jmgr:ClientNotifyURI>http://tws1.swg.be.ibm.com:31115/JobManagerRESTWeb/JobScheduler/job</jmgr:ClientNotifyURI> <jmgr:ClientNotifyURI>http://tws2.swg.be.ibm.com:31115/JobManagerRESTWeb/JobScheduler/job</jmgr:ClientNotifyURI>

© 2012 IBM Corporation25

Workload Automation

Binding process (remote engine = TWSz)• If “CP end time” < “Scheduled time of Shadow Job” ≤ “LTP end time”

Search Long Term Plan Interval searched : “CP end time” < x ≤ “Scheduled time of Shadow Job”

© 2012 IBM Corporation26

Workload Automation

Binding process (remote engine = TWSz)• If no instances are found in the LTP interval

Search Current Plan Interval searched : x ≤ “Scheduled time of Shadow Job”

© 2012 IBM Corporation27

Workload Automation

Binding process (remote engine = TWSz)• Search Long Term Plan

Matched APP1 found in LTP interval Shadow job set to BOUND

© 2012 IBM Corporation28

Workload Automation

Binding process (remote engine = TWSz)• Search Current Plan

Matched APP1 found in CP interval and it contains 002 Shadow job set to BOUND

© 2012 IBM Corporation29

Workload Automation

Binding process (remote engine = TWSz)• Search Current Plan

Matched APP1 found in CP interval and it contains 002 (but deleted) Shadow job set to SUCC

© 2012 IBM Corporation30

Workload Automation

Binding process (remote engine = TWSz)• Search Current Plan

Matched APP1 found in CP interval and it contains 002 Shadow job set to BOUND

© 2012 IBM Corporation31

Workload Automation

Binding process (remote engine = TWSz)• Search Current Plan

Matched APP1 found in CP interval but it does not contain 002 Shadow job set to ERROR

© 2012 IBM Corporation32

Workload Automation

Binding process (remote engine = TWSz)• Search Current Plan

No match found in CP interval Shadow job set to ERROR

© 2012 IBM Corporation33

Workload Automation

Binding process (remote engine = TWSz)• Search Current Plan

Instances APP1 deleted in LTP interval Shadow job set to BOUND

© 2012 IBM Corporation34

Workload Automation

Shadow job status changes

© 2012 IBM Corporation35

Workload Automation

Setup• Remote Engine

Tivoli Workload Scheduler Distributed: Nothing Tivoli Workload Scheduler for z/OS

Update ROUTOPTS in controller PARMLIB

© 2012 IBM Corporation36

Workload Automation

Setup• Local Engine

Define Remote Engine workstation Remote Engine = Tivoli Workload Scheduler Distributed

Remote Engine = Tivoli Workload Scheduler for z/OS

© 2012 IBM Corporation37

Workload Automation

Setup• Local Engine

Define Shadow Job Remote Engine = Tivoli Workload Scheduler Distributed

Remote Engine = Tivoli Workload Scheduler for z/OS

© 2012 IBM Corporation38

Workload Automation