oracle enterprise manager cloud control 12 c best practices for ... · use the transcript tab to...

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Oracle Enterprise Manager Cloud Control 12c:

Best Practices for Middleware Management

Mary Peek, Senior Principal Curriculum Developer

Hello, and welcome to this online, self-paced course entitled Oracle Enterprise Manager Cloud Control 12c: Best Practices for Middleware Management. My name is Mary Peek, and I’ll be

your guide for approximately the next two and a half hours of interactive lectures, videos, review sessions, and optional demonstrations. The goal of this course is to teach you best practices when using Oracle Enterprise Manager Cloud Control 12c for managing your WebLogic and SOA, applications and infrastructure.


Using the Player

Course Outline

Player Controls

Narration and

Best Practices

Summary

Change View

Before we begin, take a look at some of the features of this Flash-based course player. If you've attended a similar self-paced course in the past, then feel free to skip this slide. Let's start with the Outline. It's set up so that you can go at your own pace. Feel free to skip over topics you already feel confident about, or jump right to a topic that really interests you, or go back and review topics that were already covered. Just click a slide title in the outline to display its contents. By default we'll automatically walk you through the slides without requiring you to use the outline. Use the Transcript tab to view the audio transcript for each slide, and use the Search tab to find specific information in the course. Next, let's look at the Player Controls. Use these controls to Pause, Play, or move to the Previous or Next slides. You can also use the interactive Progress Bar to fast forward or rewind the current slide. Some interactive slides in this course may contain additional navigation and controls. This icon on the bottom right corner allows you to change the View of this player, for example to go to Full Screen. You can control the view at any time using this button. A downloadable, audio narration document for this course as well as the Best Practices Summary are available from the Attachments button. This course will now pause, so feel free to take some time and explore the interface. Then when you're ready to continue, click the Next button below or alternatively click About This Course in the outline on the left.

PROPERTIES

Allow user to leave interaction: Anytime

Show ‘Next Slide’ Button: Show always

Completion Button Label: Next Slide

Welcome! So, you know the title of the course, but you may be asking yourself, “Am I in the right place?”

To help you answer this question, you can access information here regarding the course objectives, the target audience, and the prerequisites. When finished, click the Next Slide button. What skills will I learn? What can you expect to get out of this course? The core learning objectives are listed here. After this course you should be able to list the most important tasks for successful management of WebLogic Server and SOA applications and infrastructure using Oracle Enterprise Manager Cloud Control 12c. You should also know when to perform each task in the middleware development lifecycle. You should be able to outline why certain management tasks are more important than others. And you should be able to provide examples of the most important management tasks. Who is the target audience? This course is intended for System Administrators and SOA Architects. What are the prerequisites? Before taking this course, you should have experience using Oracle Enterprise Manager Cloud Control 12c as well as the Fusion Middleware product stack. In Particular Oracle WebLogic Server and Oracle SOA Suite. From now on I’ll refer to Oracle Enterprise Manager Cloud Control as just Cloud Control.


Why Take This Course?

What’s in it for me?

What challenges do I face on the job?

How can this help me do my job better?

So, why take this course? Start by asking yourself, “What’s in it for me?” You will learn Oracle recommended best practices for managing your Fusion Middleware applications and infrastructure using Cloud Control. What challenges do you face on the job? Enterprise middleware applications are becoming increasingly powerful and are also more challenging to manage. How do you keep track of all your WebLogic Server configurations throughout the enterprise? How do you get visibility into your Java EE applications or your SOA composites? How do you track transactions across multiple tiers? This course will answer all those questions and more. And, finally, how can this course help you do your job better? This course shows you what tasks to prioritize in terms of middleware management so you will know what to do at each stage of the enterprise application development process. Knowing these management best practices will help your business achieve peak performance and avoid costly downtime.


Plan

Set Operational

Goals

Monitor Proactively

Diagnose Problems

Track & Control

Changes

Improve Processes

Road Map

Introducing Middleware

Management

Let's look at how this course is structured. First, I'll give you an introduction to middleware management at a high level – we'll look at the challenges you face in managing middleware

applications and infrastructure, I'll introduce Cloud Control, and I'll take you through the middleware development lifecycle which is what you see here. Then each section of the course will focus in on each stage in the lifecycle in detail.


Challenges in Middleware Management

• Monitoring multiple domains across the enterprise

• Monitoring applications across tiers

• Tracking and maintaining configurations

• Managing capacity to changing workload

• Managing multi-tier transaction flow

• Obtaining visibility into SOA services

Today’s IT organizations are increasingly adopting Java EE, SOA, composite application, and cloud computing that enable them to quickly connect disparate applications and fulfill ever changing business needs. Although these applications offer unprecedented flexibility and agility, they’re more challenging to manage.

Particularly, how do you centrally monitor multiple WebLogic domains across the enterprise? And how do you monitor your enterprise applications when they are deployed across several tiers and containers? And how do you track and maintain the configurations of numerous servers and applications? And how do you manage capacity to changing, typically increasing, workload patterns? And how do you get visibility into, and manage transaction flow that spans shared components and services and is deployed across several tiers? And how do you get visibility in to the performance of SOA and Oracle Service Bus services? And so on… Let’s hear from two Product Managers at Oracle on these challenges, first for WebLogic Server

management and then for SOA management.

Glen Hawkins, Director, Product Management for WebLogic Server discusses the challenges in WebLogic Server management.

James Kao, Director, Product Management for SOA discusses the challenges in SOA management.


Introducing Enterprise Manager Cloud Control

• Application performance management

• JVM diagnostics

• Configuration & lifecycle management

• Business transaction management

• Performance & diagnostics

• Configuration & lifecycle management

• Business transaction management

All of these middleware management challenges can lead to more business downtime, higher IT costs, and less agility. Oracle Enterprise Manager Cloud Control helps you meet these challenges. Cloud Control is Oracle's single, integrated solution for managing all aspects of the

Oracle Cloud and traditional IT environments and the applications running on them. It couples a potent, top-down monitoring approach with a cost-effective automated configuration management, provisioning, and administrative solution. This powerful combination lets you to stay focused on business priorities, while reducing the effort and cost of managing sophisticated applications built on Oracle WebLogic Server and Oracle Fusion Middleware. Oracle offers several management packs that enhance the capabilities of Cloud Control for specific purposes. In this course we'll be looking at two specific management packs – the

WebLogic Server Management Pack Enterprise Edition and the SOA Management Pack Enterprise Edition. The WLS Management pack provides application performance management, JVM diagnostics, configuration and lifecycle management, business transaction management, and more for Oracle WebLogic Server application environments. The SOA Management pack provides , performance and diagnostics, configuration and lifecycle management, and business transaction management, for a SOA-based environment. So, Cloud Control, licensed with these two packs, gives you visibility across all your middleware infrastructure and applications so you can manage them more effectively. It also reduces the cost and complexity of middleware management, thereby increasing your ROI on middleware investments.

PROPERTIES


Show ‘Next Slide’ Button: Show always


Here you can see the Middleware Development Lifecycle. Different aspects of middleware management are used at each stage of this lifecycle. This course will cover the most important middleware management best practices at each stage. Click a label for more information. Plan In the planning stage you need to look at what your middleware management requirements are in terms of what you are going to monitor and what operational procedures your organization has. Here I’ll give you best practices on what parts of Cloud Control you need to install and

where you need to install them to meet these requirements. Set Operational Goals Before you go into production you need to set your operational goals. Here I’ll give you best

practices on defining your monitoring standards. Monitor Proactively Once you are in production you need to ensure that your Oracle Fusion Middleware and WebLogic deployments are up and performing as expected. Here I’ll give you best practices on

monitoring your targets proactively so you can detect potential problems before they become severe. Diagnose Problems You need to diagnose problems at all stages in the development lifecycle – through

Development, Test, QA and production but it is vitally important in production. When your critical business applications go down or are not performing well, your business suffers. Here I’ll

give you best practices on diagnosing problems quickly and efficiently. Track & Control Changes Tracking configurations is probably one of the most time-consuming and difficult tasks you as an administrator face on a daily basis. Do you know how each of your managed targets is configured and whether they comply with your corporate or regulatory standards? Here I’ll give

you best practices on tracking and controlling configuration changes.

Improve Processes You probably spend a large amount of time performing various lifecycle management operations - such as installing, patching, and configuring WebLogic servers, as well as deploying Java applications and SOA composites, and so on. Here I’ll give you best practices

on improving your processes and automating common lifecycle management operations.


Road Map

Plan

Set Operational

Goals

Monitor Proactively

Diagnose Problems

Track & Control

Changes

Improve Processes

In the planning stage, before you even install Cloud Control, you need to look at what your Middleware management requirements are in terms of what are you going to monitor and what operational procedures your organization has. Here I’ll give you best practices on what parts of

Cloud Control you need to install and where you need to install them to meet these requirements.


Planning Best Practices

Deploy Cloud Control universally

Expand with business-driven needs

So what should you plan for in terms of installing Cloud Control to meet your middleware management requirements? First you should deploy Cloud Control universally for plug-and-play core capabilities with Management Agents on each of your monitored hosts. Then you expand with your business-driven needs. You should deploy Application, Dependency & Performance, or ADP, and JVM Diagnostics where you need deep diagnostic visibility into your applications. And you should deploy Business Transaction Management where you need visibility into your business processes.


User

Monitor & Manage Everything With One Console

DB

DB

Load

Balancer

DB

User

Cloud Control Console

Your current and planned environment will no doubt comprise many hosts running multiple WebLogic Servers, the Java EE, SOA, web applications, and business processes deployed on these servers, Oracle databases, and so on. Here you see an example production environment with multiple databases on Exadata machines, WebLogic Servers on Exalogic machines, some with a mixture of SOA and Java EE applications, Web services, Oracle Service Bus, and business processes deployed to them. A load balancer is also used to improve performance. You can, of course, use the individual product consoles to monitor the status of each of these targets, but it becomes cumbersome to shuttle between multiple console windows and track the performance of each of these targets using so many windows. Cloud Control offers a solution that allows you to monitor and manage the complete Oracle IT infrastructure from a single console. Cloud Control gives you the core infrastructure and component management as well as the metrics, configuration, lifecycle, and operational infrastructure.


Deploy Cloud Control Universally

User

DB

DB

Load

Balancer

DB

User

Management

Repository

OMSOMS

So what should you plan for in terms of installing Cloud Control to meet your middleware management requirements? First, deploy Cloud Control universally for plug-and-play core capabilities then expand with business-driven needs. The Oracle Management Service, or OMS, collects and stores data and renders it on the Cloud Control Console. In large production environments, as you see here, deploy multiple OMSs on WebLogic servers to spread the load and improve efficiency of data flow so you can meet your high availability requirements. Then you can use OMS to push Oracle Management Agents to all hosts that contain targets you want monitored. Agents can be pushed in a bulk manner and don't require manual effort for each install. An Agent is responsible for monitoring all services and components on its host. Core monitoring needs to be deployed and available everywhere so you can use it without worrying about scaling or overhead. The Agents send their monitoring data to the OMS which stores the collected information in the single management repository, a database, for future reference and analysis. When you install Cloud Control, a set of mandatory Oracle Management plug-ins are also deployed by default. These plug-ins include special management capabilities customized to suit Fusion Middleware target types and work in conjunction with the OMS and the Management Agents to monitor every Middleware target in your environment. The WebLogic Server Enterprise Edition and SOA Management Enterprise Edition Management Packs are also installed by default when installing Cloud Control.


Expand with ADP and JVM Diagnostics

User

DB

DB

Load

Balancer

DBManagement

Repository

User

JVMD

ADP

JVMD

ADP

JVMD

ADP

JVMD

ADP

JVMD

OMS

ADP

OMS

Deploy JVMD for deep

diagnostics into Java

applications.

Deploy ADP for deep visibility

into SOA, Oracle Service Bus,

Portal & ADP applications.

Once you have your Cloud Control infrastructure in place, expand it with your business-driven needs. Cloud Control also comes with some other managers and agents that are critical for middleware management. Application Dependency and Performance, or ADP, analyzes SOA, Oracle Service Bus, Portal, and ADF applications. ADP collects performance measurements, tracks contextual relationships, and summarizes data in real-time while introducing as little overhead as possible. So with ADP, you can understand the complex relationships among the business functions, associated interconnected components, and the underlying runtime environments. Deploy the ADP Manager to a managed server in the Cloud Control domain. Or multiple domains, as you see here, for load balancing and high availability. Then deploy the ADP agents on the production WebLogic servers wherever you need deeper visibility into SOA, OSB, Portal, and ADF applications. Another critical functionality in Cloud Control is JVM Diagnostics, or JVMD, which allows you to diagnose problems in Java applications in your production environment. It provides deep diagnostics into Java applications - to the Java thread level and can even provide line-of-code visibility. JVM Diagnostics can help resolve issues with applications related to java locking, memory leaks, hung threads, and so on. And it does this with near-zero overhead, making it ideal to be always-on in production to catch intermittent issues that are extremely hard to reproduce and diagnose. Deploy the JVMD Manager to a managed server in the Cloud Control domain. Or multiple domains, as you see here, for load balancing and high availability. Then deploy the JVMD agents on the production WebLogic servers wherever you need deeper diagnostic visibility into Java applications. That could include systems that are also involved in SOA, Oracle Service Bus, Portal or ADF because those are also Java based, if you need that visibility. Note, you should deploy each of the Diagnostics Managers to their own managed server with no other applications running on those servers.

So Oracle does not recommend deploying both ADP and JVMD agents universally. Only deploy them where you need that deep diagnostic visibility into your applications.


Expand with Business Transaction Management

User

DB

DB

Load

Balancer

DBManagement

Repository

User

JVMD

ADP

JVMD

BTM

BTMADP

JVMD

ADP

JVMD

ADP

JVMD

OMS

BTM

DB

ADPBTM BTM

OMS

Deploy BTM for real-time

monitoring and instance

tracking of transactions.

If you need visibility into your business transactions, further expand your Cloud Control infrastructure with Business Transaction Management, or BTM. This visibility is an important piece of middleware management as these transactions span multiple tiers and applications. BTM provides real-time monitoring and instance tracking of cross tier and cross application transactions. With it, you can non-invasively view transaction performance and conditions or faults centered on transaction performance. You can also perform payload capture, search and correlation. BTM gives you understanding of your logical transaction flow. Both the WebLogic Server and the SOA Management Packs come with BTM. In the case of the WebLogic Server Management Pack, BTM monitors Java EE services running on WebLogic Server. In the case of the SOA Management Pack, BTM monitors SOA services running on the SOA Suite. So they do not overlap. Note, however, that BTM is a separate install. The Central BTM Servers manage the BTM environment and contain the service-level and transaction management components. Deploy these three BTM server applications to a WebLogic server. They also require an Oracle database for storing performance data, logging messages, and so on. BTM Observers monitor messages and calls between the components of your applications. Decide which business transactions need to be managed then deploy BTM Observers onto the WebLogic Servers involved in those transactions. BTM Monitors collect application performance and usage measurements from observers then pass this data onto the Central BTM servers. Deploy the monitor application to a WebLogic Server.


Real User Experience Insight

Install RUEI to monitor

and manage your end

user experience.

Another management tool that you may want to install is Real User Experience Insight, or RUEI. Install RUEI to proactively monitor and manage your end user experience. You can see who your users are, where are they coming from, what they’re buying or not buying, what kind of user experience they are getting, whether they’re running into errors, etc. RUEI collects,

processes and presents every detail of your end user experience. This is a separate product from Cloud Control but there is tight integration between the two so you can drill down directly from RUEI into Cloud Control to view the deeper monitoring and diagnostics data or into BTM to view transaction data.


Planning Summary

• Deploy Cloud Control universally

• Expand with business-driven needs by selectively

deploying:

– ADP for SOA, OSB, Portal, and ADF applications

– JVMD for Java applications

– BTM for transactions

– RUEI for user experience management

Now let's summarize the best practices we covered in this planning stage of the development lifecycle. First deploy Cloud Control universally and then expand with your business-driven needs.

• Deploy ADP where you need deep visibility into SOA, OSB, Portal, and ADF applications. • Deploy JVMD where you need deep diagnostic visibility into Java applications. • Deploy BTM where you need visibility into your transactions. • And install RUEI to monitor and manage your user experience.


Road Map

Plan

Set Operational

Goals

Monitor Proactively

Diagnose Problems

Track & Control

Changes

Improve Processes

Before you go into production, you want to set your operational goals. This is where you decide what to monitor and when you want to be alerted if a target is not performing as expected. Here I’ll give you best practices on defining your monitoring standards so you can effectively monitor

the whole range of middleware targets in your enterprise.


Setting Operational Goals Best Practices

Define Metric Thresholds

Create Monitoring Templates

Create Template Collections

Create Administration Groups

First you should define your metric thresholds so you can be proactively alerted of problems before they occur. Next you should create monitoring templates to group together thresholds by target type and simplify managing and monitoring many targets. Then you should group together monitoring templates of different types to create template collections. Also you should create Administration groups to group together different target types. This is what you associate the monitoring template collections with in the next stage to further ease managing and monitoring a large number of components in your enterprise. These are the most important tasks at this stage. Let’s look at each of these in turn.


Administrator

What are Metric Thresholds?

Thresholdcrossed

AlertEmail /

Page

Metric thresholds are very important for any type of management and monitoring and, in fact, are often called the "bread and butter of monitoring". So defining these is an extremely important task. You can specify metric thresholds for each performance metric that Cloud Control monitors. A metric threshold causes an alert to be triggered when a particular performance metric value exceeds the limit. For example, when the response time on a critical application, becomes very slow. Often you're not aware of performance problems until users call to complain. Using rules and notifications, which we will discuss later, you can be sent an email and/or be paged when a threshold is crossed. This enables you to be more proactive in monitoring the availability and performance of your Middleware and the applications it supports. So thresholds let you detect impending IT problems. At this stage, you define the operational goals, that is, the thresholds, and then in the next stage, as you go into production, you'll set up what to do with the alerts by creating rules and notifications.


Metric Threshold Example

100

90

80

70

80

70

60

50

40

30

20

10

JVM

Heap

Usage

%

1 PM 2 PM 3 PM 4 PM

Critical Alert

Warning Alert

You can set metric threshold values for two levels of alert severity – warning or critical. For example, you want to monitor the JMV Heap Usage on a critical WebLogic Server. So you define a warning threshold of greater than 80%. Warning means that attention is required but the area is still functional. And you define a critical threshold of greater than 90%. Critical means immediate action is required because the area is either non-functional or there is an imminent problem. When the JVM heap usage exceeds 80% a warning alert is created and you should take corrective action. When the JVM heap usage exceeds 90% another alert is created and you need to take immediate action. Development, Test and Production systems will have different thresholds. In Development and Test you want the thresholds to be achievable so you set them based on the data you have in those environments. But you don't want to wait until you go into Production to define your thresholds. You need to set them up in Test, based on test data, then refine them as you go into production.

PROPERTIES


Show ‘Next Slide’ Button: Show upon completion


So how do you decide what to set metric thresholds on and what to set them to? Follow this 5-step procedure to define your metric thresholds. Click through the numbered steps at the bottom to proceed. Determine Critical Components To decide what components to set your thresholds on, determine your critical components involved in delivering your services. These are your WebLogic Servers, your Java EE applications, SOA composites, and so on which are all target types in Cloud Control. Choose Metrics Look at the metrics available in Cloud Control for each of the target types for your critical components and decide which are the important metrics that you need to be alerted on if they go out of normal range. For example, you may want to be alerted if your WebLogic Server ‘Datasource Failed Waiting Connection Requests’ exceeds a certain limit. And you probably

want to know if the average response time on a critical SOA composite becomes too long. Choose only the metrics you care about. You should only set thresholds for metrics whose events you will manage. You do not want to generate unnecessary metric alerts. You may also want to disable metric collection for those metrics you do not need to avoid unnecessary collection and storage of those metric values. Determine Acceptable Performance To determine what to set the threshold values to, analyze your environment during load testing and determine when performance is acceptable for a typical workload. That is when it meets your service level agreements. Define Thresholds While performance is acceptable, you can identify the threshold values for key performance indicators by viewing performance metrics in Cloud Control. Here you’re looking at your critical

business applications. You decide on the service level you require and determine your thresholds from that. When defining thresholds, the key is to choose acceptable values to avoid

unnecessary alerts, while still being notified of issues in a timely manner. Be conservative with critical thresholds – reserve these for high signals of a serious problems! Note that a few metric

thresholds come predefined out of the box. Although these values are acceptable for most monitoring conditions, your environment may require customized threshold values to reflect the operational norms of your environment accurately. Fine-Tune Once thresholds are set, you can then fine-tune these values over time until your system meets or exceeds your performance goals. And Cloud Control can also make recommendations for thresholds based on historical data, however, this would only apply once you’ve been in production for a while.


Threshold 1

Monitoring Template

Threshold 2

Threshold 3

Create Monitoring Templates

Avg Response Time > 10/20

Production SOA

Composites

Business Faults > 10/15

Errors > 5/10

Production

Servers

To actually set your thresholds, you create monitoring templates. These allow you to set one or more threshold values for a particular target type. Then later you can apply them to one or more targets of that type. For example as you see here, you want to set performance thresholds on all your production SOA composite applications. You decide that ‘Average Response Time’, ‘Business Faults’ and ‘Errors’ are what you want to monitor and you define the thresholds for

each of these. Then you create a monitoring template called Production SOA Composites and actually set the threshold values. Then you can apply these thresholds to all your production SOA Composite applications running on your production servers. This simplifies the task of standardizing monitoring settings across your enterprise because you set specific thresholds once and then apply the template to the appropriate WebLogic Servers, applications, and services. You want to avoid setting thresholds individually, server by server or application by application, instead create templates that reflect your overall business Service Level Agreements (or SLAs) and then mass apply them to your whole environment. (Here we’re talking about simple

business SLAs that translate into thresholds in Cloud Control, for example, you want response time to be less than a minute. There are also more complex SLAs that are used for service provider contracts. These map to the SLA feature in Cloud Control which is beyond the scope of this course so I won’t be going into those.)

Note that you can create thresholds at different levels of granularity. Generally you want to create broadly applicable thresholds that can be templatized. The templates are the base from which additional fine-grained thresholds may be set. You may need to add in fine-grained thresholds as you encounter specific issues, but typically this will be rare. Also Cloud Control does include predefined templates for monitoring Oracle target types, such as Oracle WebLogic Domain, Oracle Fusion Middleware Farm, Application Deployment, Oracle WebLogic Server, etc. You can use predefined templates as a starting point or create a new monitoring template.

At this setting operational goals stage, you create the monitoring templates and you apply them in the next stage as you go into production.


Production Template

Collection

Template Collection

Create Template Collections

Monitoring Templates

Threshold 1

Production

SOA

Composite

Threshold 2

Threshold 3

Threshold 1

Production

Application

Deployment

Threshold 2

Threshold 3

Threshold 1

Production

WebLogic

Server

Threshold 2

Threshold 3

Taking it a another step, you can further ease the administration of large enterprise environments by creating template collections. A template collection is a group of settings used to monitor and manage different types of targets within Cloud Control. A template collection contains a collection of different monitoring templates. You can only have one monitoring template per target type in the template collection. So for example, as you see here we have one for SOA Composites, one for Application Deployments, and one for WebLogic Servers. You could have others as well. And in this example the template collection contains the settings for production targets. Template collections can also contain compliance standards and cloud policies. But they are beyond the scope of this course and so I won’t be going into those here.


Production

Group

Production Template

Collection

Associate

Administration Group Template Collection

Create Administration Groups

The reason you create template collections is so you can associate them with Administration Groups. Today’s IT operations can be responsible for managing a great number of components, such as

WebLogic domains, SOA infrastructure, databases, hosts, or other components, which can be time consuming and impossible to manage individually. You can combine targets into logical sets, called Administration groups and then collectively apply monitoring and management settings to those groups. This enables you to organize, manage, and effectively monitor the potentially large number of targets in your enterprise. So you should put together the targets that are monitored and managed in the same way into a group – for example all your Production

systems could be monitored and managed in one way, versus your non-Production systems. Or you could group together all the targets that support the same application. Then you associate a template collection with an Administration Group so all the monitoring settings in the template collection are applied to the members of the group. Then, when a new target is added to the group, Cloud Control automatically propagates monitoring and other management settings using the Template Collection. This completely eliminates the need for administrator intervention. This level of automation simplifies the target setup process and also enables a datacenter to easily scale as new targets are added to Cloud Control. (Administrator privileges also propagate in Administration groups when new targets are added.) At this setting operational goals stage, create the template collections and, in the next stage as you go into production, you associate them with Administration groups. Oracle recommends using template collections to apply your thresholds. Applying monitoring templates directly to targets is recommended only for ad hoc operations.


Setting Operational Goals Summary

• Define metric thresholds

• Create Monitoring Templates

• Create Template Collections

• Create Administration Groups

Now let’s summarize the best practices we covered in this setting operational goals stage of the development lifecycle.

• Define metric thresholds on your critical components based on performance that meets your Service Level Agreements.

• Create monitoring templates to group together thresholds for targets of the same type. • Create template collections to group together thresholds for groups of targets of different types. • Create Administration Groups to monitor and manage different types of targets in the same way.


Road Map

Plan

Set Operational

Goals

Monitor Proactively

Diagnose Problems

Track & Control

Changes

Improve Processes

An integral part of your daily administrator duties is to ensure that your Fusion Middleware and WebLogic deployments are up and performing as expected. So once you are in production it is highly advantageous to proactively monitor your targets for potential problems before they become severe. Here I’ll give you best practices on monitoring your targets proactively.


MANAGE BY EXCEPTION

Monitoring Proactively Best Practices

Apply monitoring thresholds to your targets

Set up notifications

Create reports

Customize and build monitoring dashboards

Monitor diagnostic findings

Monitor transactions with BTM

Oracle’s recommended general best practice here is to manage by exception. You don’t have time to sit all day staring at the Cloud Control console, especially since a lot of the data is quite low level. That’s not an effective use of your time. We’ll talk more about diagnostics and when

to view low-level metrics later but for now you should set up proactive monitoring so you can be notified of impending problems. You’ve defined your thresholds for key middleware performance metrics in the Setting Operational Goals stage by creating monitoring templates and template collections, now you need to apply those thresholds to your targets. Then you should set up notifications so you’ll be alerted if any of your thresholds are violated.

You do that by doing some initial set up then creating rules and rule sets. These are the two most important tasks at this stage. But you should also create reports so you can confirm that your metrics are good. And you should customize and build specific monitoring dashboards in Cloud Control. As I said earlier, it’s not an effective use of your time to spend all day looking at the Cloud Control Console but it’s a good idea to have a few key dashboards customized to your particular

monitoring needs that you can monitor routinely. Also you want to check your important targets for diagnostic findings and view these to be alerted of potential issues. Lastly, to monitor your transactions, use Business Transaction Management or BTM.


Apply Thresholds to Targets

• Define thresholds

• Apply thresholds to targets

– Applying monitoring template to targets directly

– Associating template collection to administration group

In the previous stage, you defined your monitoring thresholds. You did this by creating monitoring templates and adding monitoring templates to template collections to make management and monitoring in large environments easier. Now you actually apply those thresholds to your targets by either directly applying monitoring templates to targets or associating template collections with Administration groups.


Apply Monitoring Templates

Deployed

Applications

SOA

Composites

WebLogic

Servers

Thresholds

Monitoring

Templates

Targets

So you've defined all your thresholds of the same target type into monitoring templates. One way you can set your thresholds is by applying these monitoring templates directly to multiple targets of the same type. Here you see how a deployed applications monitoring template can be applied to multiple deployed applications, a SOA composites monitoring template can be applied to multiple SOA composites, and a WebLogic Server monitoring template can be applied to multiple WebLogic Servers. However, Oracle recommends this only for ad hoc operations.


Associate Template Collections with Groups

Deployed

Applications

SOA

Composites

WebLogic

Servers

Thresholds

Monitoring

Templates

Template

Collection

Administration

Group

As a best practice, you should use template collections to set your thresholds. By this stage, you've grouped your monitoring templates into template collections. Each of these template collections contains only one type of each monitoring template. For example as you see here, the template collection contains one deployed application template, one SOA composite template, and one WebLogic Server template. Now you associate a template collection with an Administration group to apply all the thresholds contained in the template collection to actual targets. You'll recall that administration groups are a way to group together targets of different types that are managed and monitored in the same way, for example, all your production systems and applications as you see here. By associating a template collection to an Administration group you collectively apply all the monitoring and management settings to the group in one step rather than having to apply them to each target individually. Then, when a new target is added to the group, Cloud Control automatically propagates the monitoring and management settings using the Template Collection. This allows you to manage many targets as one and is scalable as your enterprise grows. As new targets are added there is now minimal effort to monitor those new targets. So Oracle recommends using template collections and Administration groups as the preferred way to apply your thresholds across your enterprise.


Set Up Notifications

• Define thresholds

• Apply thresholds to targets

• Set up notifications

– Set up for notifications

– Create rules and rule sets

You’ve defined your thresholds for key middleware performance metrics in the Setting Operational Goals stage by creating monitoring templates. And you’ve applied these thresholds

to your targets using template collections. Now you need to set up notifications so you’ll be notified when your warning and critical

thresholds are crossed. This allows you to be proactive in your middleware management and monitoring. You’ll know about a problem before it affects users negatively. For example, when

SOA applications are failing, proactive alerting lets you initiate the diagnostic process before your phone starts ringing. To set up notifications you need to perform some technical setup tasks and then you need to create rules and rule sets. Let’s look at these in more detail.


Types of Notifications

• OS command

• PL/SQL procedure

• Email / page Administrator

• SNMP third-party applications SNMP

Notifications are typically performed by sending an e-mail and or a page to an Administrator as you see here. For example, you may choose to email an Administrator if a warning threshold is crossed whereas you page them if a critical threshold is crossed. But you can also set up more advanced notification methods. You can execute operating system commands (including scripts), you can also execute PL/SQL procedures, and you can send SNMP traps to SNMP-enabled third-party applications. So, for example, if a critical WebLogic Server goes down, you may want to automatically open an in-house trouble-ticket using an operating system script so that the appropriate IT staff can respond in a timely manner.


Set Up for Basic Notifications

Email / page Administrator:

• Set up Mail Server

• Define Administrators’ email addresses

To set up for basic notifications, that is email and page, you need to set up your mail server and define email addresses for each administrator who needs to receive notifications. And for email notifications, you can customize the email format. You can add target properties such as Line of Business, Owner, Contact, etc in the email to provide additional operational information that can be very helpful when a notification occurs. More set up is required for the advanced notification methods which is beyond the scope of this course and I won’t go into here.

Then to start receiving notifications, you create rules and rule sets. But before we go into that, let’s talk about the different kinds of occurrences that can trigger notifications.


What can Trigger Notifications?

Performance Space Target Down

INCIDENTS

EVENTS

Metric

Alerts

Job

Events

Standards

Violations

Availability

Events

Other

Events

As you now know, when a threshold is crossed this creates a metric alert. A metric alert is one type of event in Cloud Control. An event is a significant occurrence on a managed target that typically indicates something has occurred outside normal operating conditions. For example, in addition to threshold violations creating a metric event, a database target down creates an availability event, a job failure creates a job event, a compliance standards violation creates a standards violation event, and so on. Cloud Control monitors the software stack from applications, databases to hosts and the operating system. When Cloud Control detects issues in any of this infrastructure, it raises events. You can manage events using incidents. Generally, an incident is a situation or issue you need to act on. Of all the events raised within your managed environment, there is likely only a subset that you need to act on because they impact your business applications. An incident is, therefore, a significant event (such as a target down event), or a combination of events that relate to the same issue (for example, running out of space can be detected as separate events raised from the database, host, and storage targets). You manage incidents using Incident Manager which we'll discuss later. You can set up notifications to be triggered by either events or incidents.


Targets

Create Rules to Automate Notifications

AdministratorNotificationsRule

Set

So now you need to automate the notifications related to events and incidents. This automation in support of operational processes is an important part of any scalable monitoring solution. To do so you create rules and rule sets. These enable you to choose the targets and the conditions for which you want to receive notifications from Cloud Control. Basically, a rule is an instruction for Cloud Control to take action on an event, incident, or problem. (A problem is a special type of incident raised by Cloud Control that is outside the scope of this course so I won’t cover them here.) As we saw earlier, actions include sending

email and page notifications, opening helpdesk tickets, creating incidents, and so on. To create rules you create rule sets. These allow you to logically combine different rules that operate on a common object such as a group of targets, as you see here, into a single manageable unit.


Rules Example

Rule Set for PROD-GROUP

Applies to: PROD-GROUP

Target Down Rule

Criteria: SOA Composite Average Response Time, Business Faults metric alert events of warning or critical severity

Action: Create incident, Set Priority = High

Email and Pager Rule

Criteria: All Incidents with severity= Critical or Warning

Action: If severity = Warning, email

If severity = Fatal or Critical, page

Escalation Rule

Criteria: All Incidents open > 24 hours

Action: Escalate to Level 1

For example, you can create a rule set for your production targets, that is, all the targets in your Production Administration Group. You can then create the rules. You can set a rule on a number of metric alerts, say, if SOA composite 'Average Response Time' and 'Business Faults' cross warning or critical thresholds then an incident is created. Oracle recommends monitoring and managing events using incidents. Therefore use rules to automatically create incidents for events that are important to be managed. In a rule set, the rules that create incidents should typically be the first set of rules in the rule set. There is an out-of-the-box rule set that does this which you should review to see if it meets your requirements. Then you can create another rule on incidents of warning or critical severity. If the severity is warning you can be notified by email, if the severity is critical you can be notified by page. Finally you can create another rule on incidents with critical severity. If they are open more than 24 hours you can escalate the incident to level 1. So this rule set automates a business process to create an incident, notify you of the impending problem and escalate it if no action is taken.

PROPERTIES




Here are a few tips on setting up your rules. Click on each of the segments within the circle to view Oracle’s recommendations. Rules Recall that you create rules to automate the notifications related to events and incidents. Plan You can see setting up your rules is very important so take time to plan the rule structure before implementing them. Use Groups Specify an Administration Group as the object of the rule set rather than individual targets. It’s

much easier to manage and monitor your targets in groups. The group you use in your rule set should contain targets that have common requirements for notification as well as common requirements for actions on events and incidents. As the group later expands and additional targets are added, the rule set will automatically apply to the newly added members without further modification of the rule set. Also put all the rules that pertain to an Administration group into one rule set instead of across multiple rule sets. Doing so makes it much easier to track and manage all actions on the group because it’s centrally defined in one place. And it avoids potential duplication of actions across

rule sets. Create Rules Set the rules for production groups different than the rules for non-production groups. Your production operations are critical and so you should be notified immediately if anything is going wrong before it becomes a major problem. Non-production targets are not as critical so you don’t need to have such stringent requirements. Create Incidents Create incidents for the events specified in the rule. Since there are operational requirements to send notifications for these events, then this means these events are actively managed and

thus incidents should be created for these. Rules that create incidents for events should typically be the first set of rules in the rule set. Act on Incidents Once incidents are created, perform any other subsequent actions in the rule set, such as notifications, on the incident instead of on the event. One exception would be in the case where interested parties are interested in receiving notifications for specific events. For example, business owners might want to be notified of ‘target down events’ for targets that impact their

business applications. Remind Set up reminders using repeat notifications so administrators are notified repeatedly until an important incident is acknowledged. When critical thresholds are crossed you need to ensure the problem is taken care of by an Administrator. Escalate If notifications are not being responded to, for example, the recipient may be out sick for a day, escalate these unattended, important notifications. Send a page to a different person, say at the manager level, if an incident is open too long. Review If you are receiving too many or not enough notifications, review your thresholds. You may need to fine-tune threshold values over time until your system meets or exceeds your performance goals. You can use as a guide recommendations made by Cloud Control for thresholds based on metric data for recent periods of time, for example the last 31 days.


Create Reports

You’ve set up notifications to be notified when things are changing. However, you may still want confirmation that your metrics are good. Cloud Control comes with a build in reporting infrastructure that you can use to generate reports. So to check that everything is OK, generate reports on your managed targets. For example, you may want to run weekly reports on the performance of your critical WebLogic Server, SOA or OSB performance. The report shown here shows a Summary, Availability State and Availability history for a host target over the past 24 hours. You can use predefined report definitions to generate fully formatted HTML reports, which show critical operations and business information, without any additional configuration or setup. Or you can make your own customized reports that suit your specific operational needs. And you can set up automatic email delivery so you don’t need to run them manually.


Customize and Build Monitoring Dashboards

• Proactively monitor a few summary dashboards

• Customize default dashboards

• Create new dashboards

It's also a good idea to have at least a few Cloud Control summary dashboards that are targeted to your critical applications that you can proactively monitor. These can give you a quick overview of the health, status, and performance of your middleware targets and Cloud Control provides many default dashboards to do just that. However you should customize these and create new ones to meet your specific monitoring requirements for your mission critical applications. This lets you more efficiently analyze, correlate and diagnose performance problems.


Customize and Build Monitoring Dashboards

Composite

Application

WebLogic

Server

Home Page

Middleware

Performance

Summary

Incident

Views

Group

Home

Page

SOA

Home

Page

Here you see some of the most useful dashboards to customize or build. Click on each of these dashboards to find out more details. When you are finished exploring, click Next to continue on with the course.


Customize WebLogic Server Home Page

Home pages make it easy to locate the most important monitoring data and the most commonly used administrative functions. However, you may want to go beyond the default views to see monitoring data particular to your environment. You can customize any target home page in Cloud Control but two important home pages for Middleware management and monitoring are the WebLogic Server Domain and the WebLogic Server home pages. The default view, as you see here, gives you an overview of the state of your WebLogic Servers but you may want to see additional metrics, for example, you can add particular metrics you are interested in using the Performance Metric Chart. You can also re-organize the regions displayed to help you monitor your servers more easily. Do this by personalizing the page - you can customize the format of the page as well as choosing what data to display on the page, taking off some content and adding other content.


Customize SOA Home Page

The SOA target home page is the main landing page and jump-off point for users to interact with Cloud Control functionality related to the Oracle SOA Suite. Here you see a comprehensive overview across the entire SOA Infrastructure - aggregated throughput metrics for the whole soa-infra, summary metrics for the service engines, and summaries for the SOA composites. You can use this screen to get a quick overview of the health, status, and performance of the SOA domain. Add specific additional metrics and re-organize the regions which are displayed to suit your particular needs.


Customize Group Home Page

An important home page worth customizing is the Group Home Page. When you create an Administration group to group together all your targets for a particular purpose, for example, all your production targets or all your WebLogic targets associated with a particular set of applications as you see here, you get a group home page so you can visually monitor them together. Customize this home page so you see just the monitoring information you need for your particular requirements.


Customize Middleware Performance Summary

An important Middleware monitoring dashboard you can customize is the Performance Summary page. Here you see all the relevant performance data for a particular middleware target. Using this page you can ensure your Fusion Middleware and WebLogic Server deployments are performing as expected and meeting your targets. You can customize this page to monitor exactly the performance metrics you want. Here we've added 'Deployment Work Manager Requests' to the default view. You can also select multiple metrics at a time for correlation, overlay metrics for comparison and better analysis, compare performance impact on a periodic basis, and create performance summary baselines which is like a snapshot of performance that can be used as a benchmark.


Create Composite Application Dashboards

As well as customizing existing dashboards in Cloud Control you can create new ones. In complex datacenter environments administrators need a single view of top level metrics and information related to critical business services and applications. Composite Applications are a way for you to create your own dashboards for your custom applications. This lets you build a comprehensive view representing a multi-tier composite application composed of multiple application deployments and SOA composites. You can easily include all additional components (such as databases, service buses, Coherence clusters, and other middleware and non-middleware targets). The Composite Application dashboard provides full visibility across the composite application with access to key monitoring and diagnostics regions, which can be easily customized and personalized. The overall result is a single dashboard view providing not only health information about the application, but also deeper visibility into component health and incidents at a glance. Each logical application environment in your enterprise can have its own composite application dashboard and these dashboards are fully customizable. You can personalize any of the regions to display any relevant metric.


Create Incident Views

As well as monitoring performance metrics on various dashboards, you want to monitor your incidents by creating incident views. Now that monitoring has been set up, events will be raised and incidents created for these (using the rules you created). Incident Manager, what you see here, gives you a centralized way to manage all these different incidents. It’s a central location from which you can view, manage,

diagnose and resolve your incidents as well as identify, resolve and eliminate the root cause of disruptions. On the left side are views that allow you to look at specific incidents of interest. Cloud Control comes with default views for the most common tasks, for example, there is a view for all your open incidents and problems, another for unassigned incidents, etc. Create your own custom views to filter incidents based on your specific search criteria. A good example of this is to create a view that shows all incidents for an important group of targets, for example your Production Group.


Monitor Diagnostic Findings

MetricsConfiguration

Settings

Another monitoring task you should perform regularly is to check for Diagnostic Findings on your WebLogic Server targets. The Middleware Diagnostics Advisor takes advantage of having all the configuration information – for servers, applications, and hosts – as well as all the performance metrics and component

dependency data that Cloud Control collects automatically. It analyzes the entire stack of targets from the WebLogic Server down to the database. Based on this information it generates findings, displaying all of this correlated performance management data on a single console. More importantly, instead of just providing raw metrics and configuration details, it helps you to resolve problems and improve performance by giving actual advice. For example, it can help you identify slow SQL statements as you see here or that a JDBC connection pool is causing a performance bottleneck. We'll talk more about diagnosing problems later but you can also use the the Middleware Diagnostics Advisor page to locate a potential problem by focusing on the most common finding based on the number of occurrences in the past 6 and 24 hours. Examine the summary information at the top of the finding, including the charts. Also examine the data in the Analysis Information table to understand the finding better, and also examine the related targets table to determine which targets are impacted by this finding. Respond proactively by performing the action referenced in the Suggestion column. Here it suggests tuning the SQL statements might improve performance. Note, that currently, there are no SOA specific findings for the Diagnostics Advisor.


When to Monitor Diagnostic Findings

Monitor findings during:

• Peak load time

• High CPU utilization

• High disk I/O

• High memory utilization

So when is it advisable to monitor diagnostic findings? First, monitor the findings during peak load time to check for potential problems that the load might have caused. Also, monitor findings during high CPU utilization, high Disk I/O, or high memory utilization to check for problems that might have caused the high utilization.


Monitor Transactions Using Business Transaction Management

Lastly, use Business Transaction Management to monitor your transactions. The operational health summary dashboard gives you a roll-up of key metrics for your system and transactions. The indicators will quickly tell you if something is wrong, and the associated number will tell us the actual number of issues. For example, here you can see that one service and one endpoint are down.


Monitoring Proactively Summary

• Generally, manage by exception

• Associate Template Collections with Administration

Groups

• Set up notifications

• Create reports

• Customize and build monitoring dashboards

• Monitor Diagnostic Findings

• Monitor transactions with BTM

Now let’s summarize the best practices we covered in this monitoring proactively stage of the development lifecycle.

• Generally, Oracle recommends to manage by exception. • Associate template collections with Administration Groups to quickly apply standardized

monitoring settings to many targets of different types and have them applied automatically if a new target is added to the group.

• Set up notifications by creating rules and rule sets so you’ll be notified of impending problems. • Create reports to confirm that your metrics are good. • Customize and build monitoring dashboards to meet the specific monitoring requirements of your

mission critical applications. • Monitor diagnostic findings during peak load, high CPU utilization, high disk I/O and high memory

utilization times. • Monitor transactions with BTM to view the operational health of transactions and the systems

involved in those transactions.


Road Map

Plan

Set Operational

Goals

Monitor Proactively

Diagnose Problems

Track & Control

Changes

Improve Processes

You need to diagnose problems at all stages in the development lifecycle – through Development, Test, QA and production, but it’s vitally important in production. When your critical business applications go down or are not performing well, your business suffers. Here I’ll

give you best practices on diagnosing problems quickly and efficiently.


Diagnosing Problems Best Practices

Narrow down the problem

Drill down to find the root cause

View Diagnostic Findings

Create Diagnostic Snapshots

Diagnosing problems is a very challenging area as there can be so many different types of problems with so many different causes. And middleware applications introduce a new order of complexity in tracking down failures as they are very complicated with multiple tiers within the application stack. There's no one master methodology to finding the root cause of a problem as it depends on the symptoms you’re experiencing. So for this course, I’ll be covering diagnostics

at a very high level and making general recommendations about how to use Cloud Control to solve your middleware performance problems. Oracle’s recommended approach is to first asses your notifications and/or user reports to

narrow down the possible cause of the problem to a particular part of the infrastructure, for example, is it a problem with a WebLogic server, the SOA infrastructure, or a particular SOA composite? Then, once you’ve narrowed it down, drill-down using the appropriate customized dashboard to find the root cause. You should also view diagnostic findings from the Middleware Diagnostics Advisor. And, if you’re in the middle of a critical outage but don’t have time to diagnose the root cause,

create diagnostic snapshots for later root cause analysis.


Narrow Down the Problem

1. Assess notifications / user reports

2. View customized summary pages

So first you need to narrow down the possible cause of the problem to a particular part of the middleware infrastructure. Typically you find out about a problem by either receiving an alert notification or getting a phone call from someone. This initiates the diagnostic process. Ideally alert notifications are received before the problem has escalated too far and users are starting to notice. When you get one or more notifications, you know what metric thresholds were violated so this should help you to narrow down the cause and guide you to where to start looking. For example, notification of excessive CPU consumption by a WebLogic Server would lead you to view the WebLogic Server's home page to see what applications are running on that instance and investigate further. When you are notified of a problem by a person, this can be more challenging to narrow down as the symptoms they are seeing, for example, slow response time of an application, could be caused by so many factors, many outside of the middleware layer, for example network issues. For this course we'll assume all non-middleware infrastructure issues have been ruled out. So if you don't have a clear idea of where to start, this is where your high-level customized dashboards come in useful. For example if the complaints center on the performance of one SOA composite application you can view your composite application dashboard that you created which will show all the composite application dependencies in real time. You then may be able to see that the problem is in an underlying WebLogic Server so you can then investigate further. So the notifications and/or user reports help you to form an initial hypothesis. Then you want to view the appropriate customized summary page in Cloud Control. These provide a view from the “4,000 foot” perspective and they should point you in the direction of

the problem.


Drill Down to Find the Root Cause

1. Assess notifications / user reports

2. View customized summary pages

3. Drill down into problem area

Then from your customized dashboards, drill down into the problem area to find the root cause of the problem. This root cause analysis is handled by drilling down through multiple views that are automatically discovered by Cloud Control. These detailed models of the application infrastructure allow you to maintain your context throughout the entire drill-down process from multiple entry points ensuring that you are able to quickly get at the application metrics required to quickly resolve a performance bottleneck or other application issue. The goal is to resolve the problem as fast as possible with as little impact on the overall application as possible.


Drill Down from Summary Dashboards

Composite

Application

Target

Home

Page

Middleware

Performance

Summary

Incident

Management

Console

Log

Viewer

Application

Dependency &

Performance

Business

Transaction

Management

You can drill down into deeper diagnostics from just about any page in Cloud Control. Here you see some of the most useful dashboards you can use to drill down into specific components to find the root cause of a middleware problem. Some of these you’ve seen before in this course.

Click on a dashboard to find out how you can use it to drill down for root cause analysis. When you are finished exploring these dashboards, click Next to continue on with the course.


Drill Down From Composite Application Dashboard

You’ll recall that the Composite Application dashboard, which you can create, provides full visibility across all components from the applications to the middle-tier, hosts, services, and databases that those applications rely on. This is a useful dashboard to drill down from for problem analysis. Let’s look at an example.

Here you can see a composite application comprised of multiple SOA and Java EE components that we are monitoring from a central dashboard. You can see a variety of metrics selected from each of the members including the WebLogic Server and database targets. In addition, you can see the health of the overall members and a specialized thread state table on the JVMs. • You can see that some of the JVMs have a lot of locks. The database waits are somewhat

concerning and it makes sense to explore at least the health of the JVM. A large number of locks can be a huge red flag that your application is having problems. As the JVM is the most likely culprit, we’ll drill in from that perspective while still keeping the context of the

overall Fusion Middleware and WebLogic Server target states. If we click on this MedRecServer JVM we can drill down in-context to the JVM to examine its health in more detail.

• This takes us to the JVM Performance Diagnostics page which shows both JVM and related database data. We’ve scrolled down to view the general tab where we can see Top Requests. In this case, we’re concerned with our application health, so let’s filter on a key

request such as one of the patient registration requests which is a requirement of our application. This will filter the entire page by the request allowing us to examine everything in context including all of the thread states.

• Now on the Threads tab, we can analyze our thread state over the period in question and also look at key categories in relation to those threads such as the thread state of method and request calls. At this point, we see a lot of stuck threads and in particular a lot in the Database Wait state which could indicate a database problem. Let’s click on the Live

Thread Analysis link to determine how threads are impacting each other live.

• Let’s explore this by clicking on one of the threads to drill into the root cause of why it may

be stuck in a Database Wait state which is a definite indicator of a hung application. We can see the thread stack, browse local variables, and examine how it impacts other threads. However, in this case, we want to determine the root cause of why it’s stuck in the Database Wait state in the first place. Therefore, let’s drill down to the actual database

session that is stuck by clicking on the DB Wait link. • This takes us to the database session where we can see all the details of the session that’s

being blocked or running extremely slow. Assuming Cloud Control is monitoring your WebLogic Server, JVM, and database, you can do this type of cross-tier analysis in production environments with less than 1% overhead so it won’t impact the applications

themselves. In this case, we can now see that the session is stuck within a table lock contention state. Let’s click on the Blocking ID to determine which database session is

blocking our applications session. • We can see that our composite application’s database session was blocked by a database

session kicked off from SQL Plus outside our application. This likely represents a maintenance routine or another process or application outside of ours. We can drill in even further, to see exactly what the session is doing or we could kill the session that is blocking ours (assuming that we had the necessary DBA privileges) or notify the appropriate DBA to take action and resolve the problem.

This cross-tier analysis between the JVM diagnostics and the database diagnostics allows issues in relation to JDBC calls to underlying databases to be easily diagnosed without all of the finger pointing that sometimes occurs when trying to troubleshoot issues between the middleware and database tiers.


Drill Down From Middleware Target Home Pages

The middleware target home pages are used to get a quick overview of the health, status and performance of your WebLogic and SOA targets. You can also use these pages to drill down to find the root cause of problems. For example, from the WebLogic Server home page you can easily drill into key metrics associated with the JVM of your applications in the context of the WebLogic Server domain you are analyzing for performance trends or a potential bottleneck. Here’s another example. You receive an alert notification of excessive CPU consumption by

WebLogic Server. This leads you to investigate the applications running on that instance. By using the Servlets and JSPs tab in the Most Requested section of the WebLogic Server Home page as you see here, you can quickly identify the highest volume or least responsive application. Then you can drill down and diagnose the application's servlets, Java Server Pages, or EJBs to identify the bottleneck. And, if necessary, you can drill all the way down to the JDBC/database layer. Even if you’re unfamiliar with the application design, you can quickly pinpoint problematic bottlenecks and analyze the metrics and architecture associated with them in order to facilitate a quick turnaround to get the application up and running at the required service level.


Drill Down From Middleware Performance Summary

Performance Summary pages are where you see all the relevant performance data for a particular middleware target. We looked at customizing these pages earlier to show just the metrics you’re interested in. Here you can drill down to help diagnose and resolve problems. If

you click on the metric name, you get an Additional Information popup. From here you can view a Problem Analysis page which shows other related infrastructure metrics which are affecting the metrics being analyzed. Inspect the charts for unusual increases in recorded metrics such as request processing time, CPU usage in storage units or percent utilized, number of requests per minute, Java Virtual Machine heap memory usage, or server memory usage. If you find that request processing time increased due to a high number of requests per minute, you may need to increase the capacity of your system. If the metric charts do not indicate the cause of the problem, scroll down to the Related Information pane and inspect the List of Related Targets To Analyze table for information about target health (status) and recent configuration changes. If you want to see the topology of the components for which data is being displayed, click the Related Targets Topology tab. If the List of Related Targets to Analyze table does not indicate the cause of the problem, scroll up and click a link adjacent to one of the metrics and then click Analyze Logs in the Additional Information pop-up to display log messages for the selected target and its members during the selected time period. When used together, Problem Analysis and Analyze Logs can help you inspect metrics, target status information, and logs during troubleshooting.


Drill Down From Incident Management Console

Typically you will have incidents created automatically when your critical thresholds are crossed. From the Incident Management Console you can view and manage your incidents. You can also diagnose and resolve incidents here. It’s even integrated with “My Oracle Support” for

accelerated resolution of problems. Earlier you created views to filter your important incidents. Now you can zero in on the critical ones and drill down to resolve the problem. In this example, I created a Production Incidents view, and here we need to investigate the SOA composite that is down. When we select this incident, detailed information is show in the bottom portion of the screen. In the bottom-right portion of the page we see links to the guided diagnostics and resolution area to assist you in resolving the issue. For example, if I click the View Topology link I see the topology view where I can see the health of other targets the SOA composite depends on. And I can further drill down, in-context, into relevant lower level metrics to further investigate performance behaviors.


Drill Down From Log Viewer

The Log Viewer lets you see critical, or even just suspicious, errors across all log files, including WebLogic and Fusion Middleware logs. So you can gain access to log files regardless of where they reside or what product they relate to. Here you can search and correlate messages across log files based on time, severity or Execution Context ID (ECID). Then you can use the ECID to drill down to the middleware tier for deep-dive details to help you diagnose and resolve problems.


Drill Down From Application Dependency & Performance UI

The Application Dependency and Performance, or ADP, pages within Cloud Control analyze SOA, Oracle Service Bus, Portal, and ADF applications to capture the complex relationships among the various application building blocks. ADP delivers a holistic, service-oriented view across heterogeneous environments. Here we see the ADP page for a specific Oracle Service Bus proxy service. We can see a summary of the proxy service’s performance in tabular and in

graphical form, as well as a summary of the performance of each of the pipeline nodes. We can further drill down and see individual pipeline nodes within the proxy service allowing us to diagnose problems within the service.


Drill Down From Business Transaction Management

Business Transaction Management, or BTM, gives you a central view of distributed transactions that you can drill down into. For example, two useful predefined dashboards are the Top 10 Services and Top 10 Transactions in terms of load, uptime issues, response time and faults. Here you see the Top 10 Services dashboard. These dashboards are good starting points for further analysis, since they give you a good view over the worst performers. Then you can drill into transaction content and context. This gives a unified view of transactions even across different keys, IDs, etc. Let's look at resolving an example problem with BTM. A customer, Epsilon Enterprises, has called in, complaining about some orders that don't seem to be going through and are getting lost somewhere. Since we don't know exactly where the issue is, but we do know the customer name, we'll start by searching through our messages using the customer name. We can see a few messages related to this customer and a couple stick out as having a much longer response time than the others. Let's take a look at one of them. Now that we've found a suspicious message, we can use this to look at the relevant transaction. In the transaction inspector, we can see that something is wrong with the PurchasingService. Drilling down into this service, we can now see in the response that something is wrong with the outbound service endpoint URL, most likely a typo in the service name.


View Diagnostic Findings from the Middleware Diagnostics Advisor

Earlier we talked about monitoring diagnostic findings from the Middleware Diagnostics Advisor to locate potential problems. You should also consult the diagnostic advisor real-time to resolve current problems quickly and with guided help. For example, on this WebLogic Server home page, we see Diagnostic Findings which is a clear indicator that Cloud Control has discovered a key issue with the overall server by analyzing the performance and configuration data of the entire stack including the middleware and database. If we click the Diagnostic Findings link, this starts the Middleware Diagnostics Advisor. We can see that there are at least two findings. One related to locked threads and one related to the SQL execution under the covers of our Medrec application. We can see these occurred a number of times with an hourly interval indicating it’s a regular problem that’s occurring constantly as opposed to being intermittent. Let’s take a look at the details of the findings for

this hour. If we click on the SQL finding, we see the details associated with the finding and we can see the associated metrics and configuration data that helped determine the cause of this finding. You can see that it’s found that SQL execution is taking a long time causing requests to pile up and

increasing response time and reducing throughput. And you can also see the recommendation which is to tune SQL statements to improve performance. It also lists the SQL statements themselves which are taking a significant time to run, in this case just one. You could even drill down into the top one which is the slowest, to look at the SQL in the SQL Analyzer which would allow the DBA to analyze it more fully and tune the SQL. If we look at the Thread Lock finding, we can see that there are threads in the WebLogic Server that are holding locks for a long time and other threads that are waiting and can’t proceed until

these locks are acquired. This is causing server performance to degrade. The recommendation is to drill down into JVMD diagnostics by clicking on the Lock object to determine the threads holding and waiting on these locks. And it also recommends to improve the design of the code to prevent these situations from occurring.

So the Middleware Diagnostics Advisor with it’s proactive root-cause-analysis helps you save

valuable time when it comes to identifying performance bottlenecks in production applications.


Diagnose Problems Using Diagnostic Snapshots

Quickly capture

information about

a problem.

Share snapshots

with Oracle Support

and colleagues.

Problems often arise suddenly and need to be remedied immediately with no time to capture the details for later analysis of the root cause. For example, your WebLogic Server and the underlying JVM enter some locked condition which causes your end users to hang. You need to get your users up and running as soon as possible, and so you don’t have time to figure out

why the problem occurred. You would like to keep data on the problem, and diagnose the issue later, in case it happens again. In these situations, you should immediately capture the state of the JVM, overall metrics for the WebLogic Server, and log data, as the problem occurs, into a single packaged diagnostic snapshot for later analysis. These diagnostic snapshots can then be exported and shared with others to help diagnose the root cause of the problem. Oracle Support can also use these snapshots to diagnose problems, allowing you to communicate your problems to support much easier. You can also use these snapshots to validate that a fix you put into place for a problem has in fact resolved it. This allows you to build in preventatives to better protect your environment against outages and bottlenecks in the future similar to the one that impacted your system. Note that currently there are no SOA metrics in the Diagnostics Snapshot.


Diagnosing Problems Summary

• Assess notifications/user reports to narrow down the

problem

• Drill down using customized dashboards

• View Diagnostic Findings

• Create Diagnostic Snapshots

Now let’s summarize the best practices we covered in this diagnosing problems stage of the development lifecycle.

• Assess notifications and/or user reports to narrow down the problem to a particular part of your enterprise infrastructure.

• Drill down from the appropriate customized summary dashboard to find the root cause of the problem.

• View Diagnostic Findings from the Middleware Diagnostic Advisor to resolve problems quickly with guided help.

• Create Diagnostic Snapshots in the midst of resolving problems for later root cause analysis.


Road Map

Plan

Set Operational

Goals

Monitor Proactively

Diagnose Problems

Track & Control

Changes

Improve Processes

Tracking configurations is probably one of the most time-consuming and difficult tasks you, as an administrator, face on a daily basis. For example, do you know how each and every one of your managed targets is configured and whether they comply with your corporate or regulatory standards? Here I'll give you best practices on tracking and controlling configuration changes.


Tracking & Controlling Changes Best Practices

View configuration history

Compare configurations

Create “Gold Standard” Configuration Snapshots

Customize Comparison Templates

Schedule comparisons with notifications

Cloud Control automatically collects configuration information for all the hosts and the managed targets on those hosts that have a running management agent. Using this data it lets you easily track and control configuration changes in your middleware environments. As part of configuration management you should view configuration history to pinpoint why a system may be behaving differently today that was working well yesterday. Or compare configurations of two targets that were identical to check for configuration drift. You should also create “Gold Standard” configuration snapshots that you can compare to your

live configurations. And you should customize configuration comparison templates to specify what to ignore and what to alert on. And finally you should schedule your comparisons so they run regularly and set up notifications so you're notified automatically of important differences. All of these extensive configuration management capabilities that Cloud Control provides facilitates compliance with various regulatory and business standards. So let's look at each of these in more detail.


View Configuration History

Having recent configuration changes for a particular Middleware target readily available is very valuable. For example, when an application that was performing well yesterday isn’t performing

well today, often, the performance degradation can be attributed to a recent configuration change. So if you notice a sudden change in performance, check the target home page for any Configuration Changes, as you see here. Clicking the link will show you all the configuration changes in the past week, including the old value and the new value. In this example here, we see that the login timeout setting changed from 25 seconds to 27 seconds in the past week. This can be invaluable information to help resolve problems quickly.


Compare Configurations

PRODUCTIONSTAGING

ThreadCount

MaxCapacity

StatementCacheSize

Another way to figure out why a particular middleware target is not performing as well as it was, is to compare its configuration with another target of the same type. Cloud Control’s

Comparison wizard allows you to compare configurations to quickly and easily pinpoint the differences between them. This helps to keep systems synchronized and to reduce “configuration drift”. In addition, it simplifies investigations into why systems that are presumed

to be identical, are behaving differently. A common use case for this is when you are comparing a WebLogic Server in a staging environment with another WebLogic Server in a production environment. In this scenario, there’s a performance problem with the production environment but not the staging, and you

want to compare the configurations as, originally, the two configurations were identical. Upon comparison you discover that a configuration change was made that introduced the poor performance in the production environment. For example, a key tuning parameter such as ExecuteQueue ThreadCount, JDBC Connection Pool MaxCapacity or StatementCacheSize may have been changed. And when configuration differences are detected between targets that were presumed to be identical, you can use Cloud Control to quickly provision configurations such that there are no longer any differences. We’ll look at doing this in the next section of the course.

Note, that as well as comparing WebLogic Server, domains, and cluster target types, you can also compare other middleware targets such as SOA composites, SOA Infra, JVMs, and more.


Create “Gold Standard” Configuration Snapshots

• Create snapshots of a well configured

target

• Compare current configurations to

“Gold Standard” configurations to:

– Monitor drift

– Quickly identify changes

during critical outages

An important best practice is to create “gold standard” point-in-time configuration snapshots for later analysis and comparison. For example, save the configuration of a WebLogic Server you know is configured correctly for your production environment with a label such as “Gold Standard Production WebLogic Server Configuration’.

Then at any time you can compare the current configurations of any of your production WebLogic servers with this saved configuration to determine if any configuration drift has occurred from the gold standard. This ensures that your WebLogic Servers are all configured consistently with this known reference. In fact, your IT management may request that such configuration consistency be checked on a regular basis and a summary generated that identifies any important differences. Also during critical outages, you can compare current configuration settings of the target to the gold standard and quickly identify changes that could be the root cause.


Customize Comparison Templates

When you perform configuration comparisons, use Comparison Templates. These allow you to specify which property differences to ignore, and which property differences should trigger an alert. Out-of-the-box, Cloud Control provides comparison templates for WebLogic Server, domain, and cluster target types as well as SOA Infra, JVM, and more. These out-of-the-box templates can be used as is, or as a guideline that you can customize. For example, here I’m customizing the default SOA template. It’s already set up to ignore differences between URLs in

the different environments as these are expected to be different across SOA targets. But I may be interested in tracking any changes in Audit Level from the standard so I’ll check that to be

notified on any differences. So, you should review the out-of-the-box comparison templates and customize them as necessary to meet your specific requirements. In most cases, you may only need to make a few tweaks to get what you need.


Schedule Comparisons with Notifications

Schedule

comparison

Enter email

addressAutomatic

notification+ =

Also when you perform configuration comparisons you can schedule the comparison to be run immediately or performed later. It’s a good practice to set up these comparisons with your “Gold Standard” configuration to be performed on a regular basis, for example, at the end of each

week or the first of the month. This may even be a requirement of your IT management. In addition, when performing configuration comparisons you can enter notification email addresses for who to notify if differences are found. You should enter your email address (as well as any other administrators) so you can be automatically notified. So by scheduling the comparison job regularly and entering email addresses you can be automatically be notified on a regular basis if any configuration drift occurs in your live configuration from your gold standard. Going further, you can also validate your target configurations against industry best practices, Oracle recommendations, and internal best practices by using the Cloud Control compliance standard framework. However, I won’t be going into that here as this is beyond the scope of this

course.


Tracking & Controlling Changes Summary

• View configuration history

• Compare configurations

• Create "Gold Standard" configuration snapshots

• Customize comparison templates

• Schedule comparisons with notifications

Now let’s summarize the best practices we covered in this tracking and controlling changes stage of the development lifecycle.

• View configuration history to pinpoint why a system may be behaving differently today that was working well yesterday.

• Compare configurations to check for configuration drift. • Create “Gold Standard” configuration snapshots that you can compare against your live

configurations. • Customize comparison templates to specify what to ignore and what to alert on. • Schedule regular comparisons with automatic notifications of important differences.


Road Map

Plan

Set Operational

Goals

Monitor Proactively

Diagnose Problems

Track & Control

Changes

Improve Processes

You probably spend a large amount of time performing various lifecycle management operations - such as installing, patching, and configuring WebLogic servers, as well as deploying Java applications and SOA composites, and so on. These are very important tasks but can be very time-consuming and error-prone. Here I’ll give you best practices on improving

your processes and automating common lifecycle management operations. This will allow you to perform these tasks much faster and more reliably.


Improving Processes Best Practices

Clone Middleware components from the Software Library

Create Gold Images

Create WebLogic Provisioning Profiles

Customize default Deployment Procedures

Automate the application of WebLogic Server patches

Rather than installing or deploying Middleware components manually, which can be error-prone and introduce inconsistency in your environment, you should clone them using Cloud Control. And the best way to do that is to clone from Cloud Control’s Software Library. This ensures

consistent, standardized images and components are deployed across your environment. You should also create gold images of tested Middleware Homes and SOA Artifacts in the software library for later provisioning. As well as creating WebLogic Domain provisioning profiles to act as templates for WebLogic domain provisioning. You should also customize the default Cloud Control deployment procedures to meet your unique provisioning requirements. And lastly, you should automate the application of WebLogic Server patches to make this whole process simple, reliable, fast, and one you can do without any downtime! You can automate all these routine tasks using Cloud Control’s Software Library and

Provisioning Framework. This lets you improve quality of service by minimizing downtime due to configuration change. And it reduces your IT operational cost through automated deployment procedures to provision software. Let’s look at each of these in more detail.


• Manual installations are slow

and error-prone

• Cloning is fast and eliminates

errors

Admin Server

Cluster

WebLogic Domain

Clone Middleware Components

Admin Server

Cluster

WebLogic Domain

Administrators often rely on manual installation methods in order to provision new middleware environments or add capacity to existing servers which support their production applications. However manual installation methods can take too long and are error-prone. A much more efficient means to provision new middleware environments and add capacity is to clone middleware components in Cloud Control. Cloning reduces time and eliminates errors in building environments. You can clone directly from a source domain to create a new domain or scale up or out an existing domain. “Scaling up” refers to adding managed servers to a domain on existing hardware, while “scaling out” means adding managed servers to a domain on new hardware.

You would add capacity to existing domains or clusters to accommodate an increase in load. For example, if your deployed application is incurring high load and not performing adequately then you need to scale up or out to improve application performance. You can also clone directly from a source domain to deploy Java EE applications, or BPEL processes or from a SOA domain to provision SOA artifacts such as SOA composites, web services policies, JPS policies, and so on. You can also clone the binaries in the Middleware home directory.


Clone Middleware Components from the Software Library

• Ensure consistent, standardized images are deployed

• Rapidly scale out or revert to known good state

• Simplify lifecycle management

Software Library

Admin Server

Cluster

WebLogic Domain

Admin Server

Cluster

WebLogic Domain

You can clone domains and their components directly from an existing domain, but Oracle recommends cloning from Cloud Control’s software library to ensure consistent, standardized

images and components are deployed across your environment. This also allows you to rapidly provision middleware environments or quickly revert to a known good state. For example, you may need to scale out an application on another environment very quickly after a new version is introduced that fixes a bug or revert an application to a known good state when a problem is encountered. Cloning Middleware components from the software library simplifies lifecycle management. The software library serves as a central repository for metadata and binary content of application software, software patches, reference gold images, etc. Here you can store various revisions of WebLogic Server domains (both the binaries and the domain configuration), Middleware home binaries, Java EE applications, SOA Artifacts, BPEL processes, Oracle Service Bus Resources, Java Platform Security configuration, and so on. You can create software library components in the context of your reference domain. And, when you clone a component, for example a WebLogic domain, you can make changes to the destination domain configuration as necessary, such as host names, ports, and so on. So you should clone components from staging or reference servers into the software library then deploy those components centrally from the software library to new hardware.


Create Gold Images

Gold images are:

• Based on corporate standards

• Compliant and tested

• Stored in the Software Library

Create Gold Images for:

• Middleware Home

• SOA Artifacts

Gold images are created by component designers based on corporate standards from reference deployments. They are compliant and tested images, stored in the software library, from which they can be used by all operators in a consistent manner. You can create gold images for Middleware Homes and SOA Artifacts. Middleware Home Gold Images contain only the WebLogic binaries. SOA Artifacts Gold Images contain SOA Artifacts such as SOA Composites, Oracle WebLogic Server Policies, Assertion Templates, JPS Policy and Credential Stores. So create these tested gold images in the software library for later provisioning.


Create WebLogic Domain Provisioning Profiles

Middleware

Home Binaries

Domain

Configuration

Provisioning

Profile

+ =

Create a Template for WebLogic Domains

You should also create WebLogic Domain Provisioning Profiles from existing reference domains. A Provisioning Profile consists of both the Middleware Home binaries and the domain configuration. You create a profile, save it in the Software Library, and then use the saved profile as the source for creating new WebLogic domains. This will ensure that future WebLogic installations follow a standard, consistent configuration. It also saves time and reduces the risks of making mistakes. You can think of the provisioning profile as a template for WebLogic domains provisioning.


Customize Default Deployment Procedures

Deployment

ProcedureDeployment

ProcedureDeployment

Procedure

Add environment variables

Add custom steps

Disable unwanted steps

Add custom scripts

Use authentication tools

Use different error handlers

Email notifications

To actually perform provisioning you use a deployment procedure. This contains a hierarchical sequence of provisioning or patching steps that need to be performed for a particular lifecycle management activity. Cloud Control comes with a set of default Deployment Procedures that help you accomplish common provisioning and patching-related tasks. These default procedures have been created considering all the best practices in the industry. And each Deployment Procedure is unique, and is designed to perform a particular operation according to the source being provisioned. For example, the Deployment Procedure to deploy a Java EE application differs from the one to deploy a SOA composite. Understandably, your provisioning and patching-related requirements will vary from one environment to another, or from a test installation to a production installation. So use the default Deployment Procedures and customize them according to your needs. For example, you can add environment variables, include additional custom steps, disable unwanted steps, add custom scripts, and use authentication tools to run some steps as another user. You can even implement different error handling methods. And you can reference any of the Middleware entities in the software library to automate the patching, provisioning or deployment of the associated software. To further automate provisioning, set up email notifications to report the status of deployment procedure execution so you know immediately if it was successful or not, or action is required.

PROPERTIES




Here you see a list of the default Middleware deployment procedures that you can customize to meet your needs. Click the tabs to your left to get an overview of each deployment procedure. Provision Middleware This procedure clones and configures a Middleware Home and/or WebLogic Domain from the Software Library. Scale Up/Scale Out Middleware This procedure clones existing WebLogic Managed Servers within a domain or adds new managed servers to an existing domain in order to add capacity to the domain or cluster. Use this procedure to scale up on existing hardware within the domain or scale out on new hardware. Deploy/Undeploy Java EE Applications This procedure deploys, redeploys, and undeploys Java EE applications to and from WebLogic domains. You can deploy to multiple servers or multiple domains and you can also specify the archive files to deploy, any pre or post deploy WLST scripts, and so on. SOA Artifacts Provisioning This procedure clones and configures SOA Artifacts such as SOA composites, Web Service policies, Java Platform Security configuration, Human Workflow configuration and B2B configuration from a reference SOA Domain environment or gold image to a destination SOA Domain. Deploy SOA Composites This procedure deploys SOA Composites on a selected SOA domain. The composite and the plan can be selected from a file system accessible from the destination domain host or from the software library. BPEL Process Provisioning

This procedure deploys BPEL Processes on a selected Oracle BPEL Process Manager. To do so, select BPEL Process suitcase files which are BPEL process JAR files stored as generic components in the Software Library. Oracle Service Bus Resource Provisioning This procedure deploys Oracle Service Bus resources to an Oracle Service Bus Domain. You can deploy the resources as part of a project or individually. Resources include Business Services, Proxy Services, WS-Policies, WSDL schemas, XQuery transformations, and JARs.

PROPERTIES




Here you can explore the recommended workflow for provisioning WebLogic domains. Click the steps along the bottom to view more details. • In this first step, you manually install the WebLogic software, patch WebLogic, configure

the WebLogic domain and then fully test the environment until you consider it a ‘reference installation’ that all future installations should conform to.

• In this step you create a WebLogic Domain provisioning profile of the reference installation; where the WebLogic binaries and the domain configuration are stored in the software library for use as the source for future cloning operations.

• In this step you use the provisioning profile to create a customized deployment procedure. Here you can set new inputs, like ORACLE_HOME, ORACLE_SID, and other configuration parameters. This prevents you from having to repeat identical configuration inputs for each deployment procedure. You can also lock the procedure so that any operators running the procedure cannot modify it. This deployment procedure is now a best practice procedure.

• In this step you use the best practice deployment procedure to clone WebLogic domains with a standard, consistent configuration quickly and without errors.


Automate Application of WebLogic Server Patches

Patch Plans

• List patches to apply

• Specify targets

• CC performs validation checking

• Specify deployment procedure

Patch Templates

• Are created from Patch Plan

• List patches to apply + deployment options

• Do not specify targets

Patching is an important phase of the WebLogic Server lifecycle that enables you to keep your WebLogic Servers updated with Oracle bug fixes. However, patching has always been a very challenging phase of the lifecycle because it is complex, risky, time consuming, and involves downtime. With Cloud Control you can easily automate the application of patches across WebLogic Servers. You can automate applying one-off patches and critical patch updates across WebLogic domains by creating Patch Plans. These plans help you create a consolidated list of patches you want to apply as a group to one or more WebLogic targets. As you create the plan and specify the actual targets, Cloud Control performs prerequisite checks and analyzes the patches for conflicts and potential problems. This automated validation checking minimizes errors. Also when creating the patch plan you specify the deployment procedure to use. WebLogic Server has two default deployment procedures for patching, which of course you can customize to meet our needs. The default plans let you apply patches in either parallel or rolling mode. Applying patches in rolling mode eliminates downtime as the nodes of the cluster are patched individually, that is, one by one. For example, if you are patching a cluster that has five nodes, then the first node is shut down, patched, and restarted, and then the process is rolled over to the next node until all the nodes are patched successfully. In contrast, when patching in parallel mode, all the nodes are shut down and the patch is applied on all of them at the same time. You can save a patch plan as a patch template. This template contains the predetermined set of patches and deployment options saved from the source patch plan, but without the targets specified. This can then be used to create new plans to roll out already tested patches to your enterprise. So automating the application of patches with patch plans and patch templates makes the whole process simple, reliable, and fast, and avoids downtime!

PROPERTIES




Here you can explore the recommended workflow for automating the application of WebLogic Server patches. This process allows you to create predesigned plans based on an existing successfully analyzed and deployable patch plan which you use to select a completely new set of targets. Click the steps along the bottom to view more details. • In this first step, identify and download WebLogic Server patches through My Oracle

Support which is integrated into Cloud Control. • In this step, create a patch plan that contains all the patches you wish to apply and the

targets to apply them to. If you’re rolling out patches in a mass scale over a period of time,

apply the patches on a few WebLogic Servers to test if the patches are being applied successfully. Cloud Control validates the patches can be applied to those targets. Also select the deployment plan to use and then execute the deployment plan to apply the patches.

• In this step, save the tested patch plan as a patch template which contains everything in the patch plan except the targets.

• In this step, create patch plans from the patch template to roll out the same set of tested patches to the rest of the WebLogic servers in your enterprise.


Improving Processes Summary

• Clone components from the Software Library

• Create Gold Images

• Create WebLogic Provisioning Profiles

• Customize default Deployment Procedures

• Automate application of WebLogic Server patches

Now let’s summarize the best practices we covered in this improving processes stage of the development lifecycle.

• Clone Middleware components from the software library to ensure consistent, standardized images and components are deployed across your environment.

• Create Gold Images of tested Middleware Homes and SOA Artifacts in the software library for later provisioning.

• Create WebLogic Provisioning Profiles to act as templates for WebLogic domain provisioning. • Customize default Deployment Procedures to meet your unique provisioning requirements and

add automatic status notification. • Automate the application of WebLogic Server patches by creating patch plans and patch

templates to make the process simple, reliable, and fast and avoid downtime.

PROPERTIES




This self-study covered the most important Middleware management best practices at each stage of the middleware development lifecycle. Click a label for the Best Practices at each stage of the lifecycle. You can also download a summary of the Best Practices by clicking the Attachment link. Plan

• Deploy Cloud Control universally and then you expand with your business-driven needs. • Deploy Application Dependency & Performance where you need deep visibility into

SOA, Oracle Service Bus, Portal, and ADF applications. • Deploy JVM Diagnostics where you need deep diagnostic visibility into Java

applications. • Deploy Business Transaction Management where you need visibility into your

transactions. • And install Real User Experience Insight to monitor and manage your user experience.

Set Operational Goals: • Define metric thresholds on your critical components based on performance that meets

your Service Level Agreements. • Create monitoring templates to group together thresholds for targets of the same type. • Create template collections to group together thresholds for groups of targets of different

types. • Create Administration Groups to monitor and manage different types of targets in the

same way. Monitor Proactively:

• Generally, Oracle recommends to manage by exception. • Associate template collections with Administration Groups to quickly apply standardized

monitoring settings to many targets of different types and have them applied automatically if a new target is added to the group.

• Set up notifications by creating rules and rule sets so you’ll be notified of impending

problems. • Create reports to confirm that your metrics are good. • Customize and build monitoring dashboards to meet the specific monitoring

requirements of your mission critical applications. • Monitor diagnostic findings during peak load, high CPU utilization, high disk I/O and high

memory utilization times. • Monitor transactions with BTM to view the operational health of transactions and the

systems involved in those transactions. Diagnose Problems:

• Assess notifications and/or user reports to narrow down the problem to a particular part of your enterprise infrastructure.

• Drill down from the appropriate customized summary dashboard to find the root cause of the problem.

• View Diagnostic Findings from the Middleware Diagnostic Advisor to resolve problems quickly with guided help.

• Create Diagnostic Snapshots in the midst of resolving problems for later root cause analysis.

Track and Control Changes: • View configuration history to pinpoint why a system may be behaving differently today

that was working well yesterday. • Compare configurations to check for configuration drift. • Create “Gold Standard” configuration snapshots that you can compare against your live

configurations. • Customize comparison templates to specify what to ignore and what to alert on. • Schedule regular comparisons with automatic notifications of important differences.

Improve Processes: • Clone Middleware components from the software library to ensure consistent,

standardized images and components are deployed across your environment. • Create Gold Images of tested Middleware Homes and SOA Artifacts in the software

library for later provisioning. • Create WebLogic Provisioning Profiles to act as templates for WebLogic domain

provisioning. • Customize default Deployment Procedures to meet your unique provisioning

requirements and add automatic status notification. • Automate the application of WebLogic Server patches by creating patch plans and patch

templates to make the process simple, reliable, and fast and avoid downtime.

PROPERTIES




More Information Oracle offers a variety of additional channels from which you can learn more about Enterprise Manager Cloud Control or about any Oracle products. We at Oracle Education know your time is valuable and limited, and so we thank you for participating in this self-paced training. We hope this course has met your expectations and learning objectives, and wish you the best of luck in all of your endeavors. Please read through all the tabs on this page to learn about all the Oracle resources available for you. Self-Study Courses Oracle Enterprise Manager Cloud Control 12c Self-Study training is aimed specifically at administrators who are familiar with the current releases of Enterprise Manager. It is designed to bring existing users up to speed on the new features of Oracle Enterprise Manager Cloud Control 12c. The series is broken into a number of tracks, each based around one of the major themes of the new release, and is available from Oracle University. Instructor-led Courses Several instructor-led courses are also included in the Oracle Enterprise Manager Cloud Control 12c curriculum, available through Oracle University. Demos Many free demonstrations on Oracle Enterprise Manager Cloud Control 12c are available through the Oracle Learning Library. Here’s a list of the demonstrations that are most relevant

to this self-study. Other Resources The Oracle Learning Library offers many free demonstrations and tutorials. And, of course, the Oracle Enterprise Manager Cloud Control 12c documentation and online help embedded within the product are also valuable resources.

oracle enterprise manager cloud control 12 c best practices for ... · use the transcript tab to...

Documents