customer case study: centerpoint energy - how to achieve .0003 abends!

Customer Case Study: CenterPoint Energy - How to achieve .0003 abends!

Tess Dowdy

DevOps: Continuous Delivery

• Centerpoint Energy

• Technology Operations Manager

• DO4X180S

Customer Case Study:

CenterPoint Energy - How to achieve .0003 abends! Tess Dowdy – Technology Operations ManagerCenterPoint Energy

© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.

The content provided in this CA World 2015 presentation is intended for informational purposes only and does not form

any type of warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy

by CA.

For Informational Purposes Only

Terms of this Presentation

Abstract

Join this session to learn about some tips,

techniques, and tools that Centerpoint Energy

uses with CA Workload Automation AE

(AutoSys) to reduce the number of job failures

and reduce costs for batch processing. During

this session, Tess Dowdy/Centerpoint Energy

will also review their processes and procedures

to streamline your batch and to possibly

eliminate the late night telephone calls to the

clients.

Tess Dowdy

Centerpoint Energy

Technology Operations

Manager

CenterPoint Energy Proprietary and Confidential

Agenda

5

Who is CenterPoint Energy?

Workload Automation in our environment/Job Statistics

Reduce costs for batch processing

Tips, Techniques, and Tools

Processes and Procedures

Q&A


CenterPoint Energy (CNP)

6

Electric Transmission

& Distribution

Natural Gas Distribution

Competitive Natural Gas

Sales & Services

Headquartered in Houston, Texas

Serving 5.5 million electric & gas metered customers

$21.8 billion in assets

$8.1 billion in revenue

More than 8,700 employees

Over 135 years of service to our communities

Electric transmission and distribution delivery business

Over 2.2 million metered customers in Houston area

17.3 GW peak demand

79 GW hours delivered annually

233 substations

3,739 miles of transmission

49,162 miles of distribution


Workload Automation in our Environment

7

Application Servers (Prod & Test)

(1) CPU each

Autosys11.3.5 INCR3

AIX version 7.1

Oracle Database v11.2.0.3.7

12GB Memory (DB&APP)

3.5GB (Current Database size)

Utilization

70% AVG CPU

80% AVG Memory


Workload Automation in our Environment

8

Platforms

UNIX/LINUX

Windows

Mainframe

Agents Installed

397 servers

6 mainframe

Jobs Defined

14,530 (Prod Servers)

5,507 (Prod Mainframe)

Jobs/Failures per month

2,316,759 jobs executed

776 Failures (.0003)


Reduce Costs for Batch Processing

9

Reduce the time required to process the work

Service Level Agreements

Eliminate costly reruns

Dependencies or Negative Dependencies

Stabilize the batch processing environment

Eliminate orphaned PIDs, maintenance windows, etc.

Pinpoint bottlenecks

Utilize a critical monitoring software

Automate schedules and manage batch processing cycles

Job sets vs single jobs

Automate…. Automate…. Automate….


Tips, Techniques, and Tools

10

Tips

Utilize a test environment

Work with clients to test executable scripts

Provide recommendations/suggestions

Techniques

Agent verifications using “Touch” jobs

Utilize Machine Offline/Online jobs

Define requirements (max/min run times)

Utilize dependencies/negative dependencies

Tools

Critical Path Monitoring software

Job Scheduling form

Job Documentation database

Event management software

Change, incident, service request software


Tools – Job Scheduling Form

Blue fields must be completed or the form will be returnedJob Scheduling Form

General Information: AE11.3 v1

Application ID (example CCS, ESD, TMH)

Criticality Critical - see instructions on line 5

Special Instructions/Restart/Recovery:

Is this request a(n):

Start Date:

Requestor info

Requestor Name:

Contact Number:

Alarm Point - ServiceNow Group Name:

WBS:

CM or SR Number:

Has the job been added to the ESP Interface: NOTE: If this step is not

completed it will cause a delay in adding the job to AUTOSYS

Use this link to add the job to ESP

https://esp:8443/login.jsp

Primary Information

Job Name:

Job type:

Description:

Owner:

Dependency type: Dependency job name(s):

Jobset:

Command:

Prod send to Machine: If New box describe:

Test send to Machine: If New box describe:

Schedule

Run Calendar or

Run Frequency If Other describe:

Time(s) of day or

Minutes after each hour

Exclude Calendar

Run Window

AlarmsMust start

Must complete

Minimum run time ? minutes

Maximum run time ? minutes

Send Alarm On Failure: 1

Terminate after running ? minutes: If Yes how many minutes:

Permissions

Automatically restart Job ? times after failure:

Maximum exit code for success:

Command Info

Standard input:

Standard output: If Other describe:

Standard error: If Other describe:

Job environment/profile

11

https://esp:8443/login.jsp


Tools – Event Management

12


Tools – Event Management

13


Processes and Procedures

14

Processes:

Change Record

Test Environment

Job Scheduling Form

Job Set or Single job

Event Management

Move to Production

Procedures:

Move to Production

Critical/Non Critical

On-call notification

Service Request for action

Research job failures

Contact client for further discussions

Thank You on Behalf of CenterPoint [email protected]

For More Information

To learn more, please visit:

http://cainc.to/Nv2VOe

CA World ’15

http://cainc.to/Nv2VOe

http://bit.ly/1wbjjqX