TCP/IP Chargebacks

Nalini ElkinsInside Products, Inc.nalini_elkins@inside-products.comFebruary, 2004 SHARE Technology ExchangeSession 3744

Why bill?Why bill?

• Many companies are curious about billing forTCP/IP usage on the z/OS platform to users orapplications for their usage of associatedresources.

• TCP/IP started tasks and the services which relyon TCP/IP (such as FTP, Tivoli Service Manager,MQSeries, Websphere) use scarce and expensivehost CPU time.

Using Scarce ResourcesUsing Scarce Resources

• Often applications are coded with seemingly littleregard for the amount of CPU usage, networktraffic or other scarce resources.

• TCP/IP resource usage seems to be growingexponentially.

• A mainframe vendor cites: cutting 5% of z/OS CPUutilization can save as much as $7 million over 5years.

What Should be Billed?What Should be Billed?

We will discuss topics such as:

• What are the factors which may be monitored andbilled?

• Should peak time usage and other methods beused to help in conservation of critical resources?

• What is "fair?"

• What is practical?

• A possible methodology.

Be Able to ExplainBe Able to Explain

• You should be able to explain what you aredoing when the “customer” asks how he isbeing billed.

• Charges should be repeatable from month tomonth. CPU time sometimes has a problemwith this. Customer should be able to budget.

• Keep it simple.

CPU Time Repeatable?CPU Time Repeatable?

• CPU time may vary between different runs of the same job or job step.• Cycle stealing on systems with integrated channels -- CPU instruction

processing is temporarily suspended when channels require the use ofhardware resources shared with the CPU.

• Buffer interference caused by concurrent tasks• Partial or full disabling of a buffer because of storage errors• Storage access -- The CPU cannot access central storage if a channel is using

it. Storage-access time depends on CPU architecture such as interleaving, datawidths and paths.

• Temporary I/O errors -- Additional processing may be required for temporaryI/O errors.

• Page stealing -- Page stealing affects the number of page faults that a particularjob incurs. CPU time varies depending on both the number of page faultsresolved by I/O and the number of page faults resolved by reclaim.

• Etc….

Economics 101Economics 101

• Peak time usage can be reduced by charging for it

• Example: Telephone company

• Day, night, weekend rates curb usage

What are the scarce resources?What are the scarce resources?

• CPU time• DASD• Real / virtual storage• Network bytes• VTAM buffers• CSM

What Do We Need to Bill?What Do We Need to Bill?

• Who : The Consumer of the resource (User, Job name etc)

• When :A timestamp for when the resource was used

• Where:The z/OS platform, TCP/IP stack name, local host

• What:Which resource(s) were used and how much of each

• Which:Which services or applications were used

How: MVS SMF RecordsHow: MVS SMF Records

• SMF - Type 30– z/OS or OS 390– Job Start– Step End– Job End– Interval– Storage– CPU– OMVS

Trick is to integrate TCP/IPand MVS information!

Data SourcesData Sources

• OS 390 - Type 118– TCP Initiation– TCP Termination– FTP Server– FTP Client– Statistics

• z/OS - Type 119– TCP Initiation– TCP Termination– FTP Server– FTP Client– Statistics– Server– Interface

More data in each record

What is Available: CPU UsageWhat is Available: CPU Usage

CPU usage report shows TCB, SRB and other CPUtime in hundreds of seconds for all TCP related systemtasks as well as all socket applications.

DASD / DASD / EXCPsEXCPs

I/O analysis report shows total EXCP’s, reread countand pending time. Pending time can lead to poorresponse time.

StorageStorage

Paging analysis report shows if page swapping, stealinghas occurred. This can lead to poor response time.

Network UsageNetwork Usage

Network usage includes number of connections, number of bytes, and connect time.

The Devil is in the Details...The Devil is in the Details...

• What units are we measuring in?

• CPU time, etc is by address space. How do you bill foran address space?

• Network usage is by address space / remote host IPaddress.

• Do we need user ID? (Consider DHCP. Proxy servers)

CPU Time Charge BackCPU Time Charge Back

• Can’t allocate CPU time back to a particular user orremote ID

• Only available by socket application or ID

• Charge for what you can and allocate other portions foroverhead.

Conservation of ResourcesConservation of Resources

• Reduce, reuse, and recycle.

• What can be done?

• Complications

What Uses Resources for TCP/IP?What Uses Resources for TCP/IP?

• Many small connections vs. few long (charge for # ofconnections?)

• CPU using applications (charge for TCB / SRB time?) -can impact all users (IP discards)

• Network using users (charge for network bytes?)

How to find out who to bill?How to find out who to bill?

• Bytes & connections possible by user id or remote IPaddress.

• Charge by # of connections rather than total connecttime.

Possible Complication...Possible Complication...

• Federal agency in thegovernment of a largecountry.

• All traffic goes through 2proxy servers

Problem DefinitionProblem Definition

• Third party package forcesshort connections.

• Over 7,000 connections perminute.

• Should bill by user id?

Proposed methodologyProposed methodology

• Pay for using applications proportionately by how muchthey are used (Application Service Unit: ASU)

• User pays for # of bytes sent, # of connections for eachapplication + some for overhead.

Step 1: Calculate Application OverheadStep 1: Calculate Application Overhead

• Each TCB or SRB is x cents.• Each 1000 EXCP is x cents• Each meg of storage is x cents

• Each socket app uses certain # of TCB, SRB’s per day• Some applications have to be “overhead” applications

Add complications:• Can charge applications differently• Can charge by time of day differently

Step 1: Sample Application UseStep 1: Sample Application Use

• Each TCB or SRB is 10 cents.• Each 1000 EXCP is 20 cents• Each meg of storage is .001 cents

• Application CICSB uses 200 TCBs, 300 SRBs, 10,000 EXCPsand 2 meg of storage.

• TCB/SRB charge: 500 * 10 cents = 5,000 cents or $50.00• EXCP charge: 10,000 * 20 cents = 20,000 cents or $200.00• Storage charge: 2,000,000 * .001 = 2,000 cents or $20.00

Step 2: Calculate Total Application UseStep 2: Calculate Total Application Use

• Add up TCB or SRB charges +• EXCP charges +• Storage charges

Sample for CICSB:• TCB/SRB charge: $50.00 +• EXCP charge: $200.00 +• Storage charge: $20.00• Total : $270.00

Step 3: Calculate Overhead UsageStep 3: Calculate Overhead Usage

• Calculate amounts for address spaces used by everyone:

• TCP/IP, BPXOinit, etc

• These will added to the total

• Can do it by address space or by number of connections

Step 3: Sample Overhead UseStep 3: Sample Overhead Use

• Each TCB or SRB is 10 cents.• Each 1000 EXCP is 20 cents• Each meg of storage is .001 cents

• TCP/IP uses 1000 TCBs, 1000 SRBs, 100,000 EXCPs and 4 megof storage.

• TCB/SRB charge: 2000 * 10 cents = 20,000 cents or $200.00• EXCP charge: 100,000 * 20 cents = 2,000,000 cents or $2,000.00• Storage charge: 4,000,000 * .001 = 4,000 cents or $40.00• Total: $2,240.00

Step 4: Calculate ASUStep 4: Calculate ASU

• Application Service Unit = Application + Overhead charges

Sample for CICSB with 10 applications:• Total Application charge: $270.00 +• Proportionate Overhead Charge: $2,240 / 10 ==> $224 +

• ASU = $494

Step 5: Calculate Usage by UserStep 5: Calculate Usage by User

• Each connection to an application is ASU / totalconnections

• Each 1,000 bytes is x cents

Add complications:• Can charge by time of day differently

Step 5: Sample Usage by UserStep 5: Sample Usage by User

• User Jane has 100 connections to CICSB• CICSB has 2,000 connections over the day total• ASU for CICSB is $494• Each connections is 494 / 2000 or 25 cents

• User Jane is charged 100 * .25 or $25.00• Each 1,000 bytes is x cents (based on network hardware

calculations)

• Can charge by time of day differently

Side note: How to Decide What theSide note: How to Decide What theNumbers Should Be?Numbers Should Be?

• That is the trick, isn’t it?

• Add up hardware, software costs

• See what percent of usage is due to TCP/IP tasks

Case Study of How Billing ChangedCase Study of How Billing ChangedUsageUsage

1. Set Goal. Example: For Friends Provident it was:

Reduce the peak usage on the mainframe by 100MIPS. Any changes you make should not impactanyone’s ability to do their work.

2. Measure workload configuration and usage. (Hardware, software, how many MIPs)

Sample Workload and MIPS ProfileSample Workload and MIPS Profile

Calculate the CostsCalculate the Costs

3. Calculate the costs. How much does it cost to use 1MIPs worth of work for 1 year?– Include hardware, IBM software, and ISV software.– Their sample costs were 3,500 pounds per MIP.– See about peak vs. off peak charging.– Friends Provident charged double for peak hour

usage.

Examine the WorkloadExamine the Workload

4. Examine the workload– See exactly what is running at peak times– How much is being used.– Concentrate on heavy hitters

5. Solutions– Rewrite more efficiently– Move work to off-peak– Capping (use resource groups in WLM)– Drop unnecessary usage

Provide IncentivesProvide Incentives

6. Provide incentives– Better service off-peak (higher priority, more

initiators)– More consistent service (smooth out peaks,

congestion)– Automation (to run batch workloads off-peak)

Good Luck!Good Luck!

• Try methodology tosystematically recover costs

• Good luck!