oracle trace file analyzer (tfa) - aioug customer slides.pdfprovides health checks for the oracle...

69
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Trace File Analyzer (TFA) 12.2.1.1.0

Upload: leanh

Post on 20-Apr-2018

228 views

Category:

Documents


5 download

TRANSCRIPT

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Oracle Trace File Analyzer (TFA) 12.2.1.1.0

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Understand TFA & install

Basic configuration

View system & cluster summary

Investigate logs & analyze

Collect diagnostics

Manage database logs

Customize collections

Advanced configuration

1

2

3

4

5

3

7

6

8

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Understand TFA & install

Basic configuration

View system & cluster summary

Investigate logs & analyze

Collect diagnostics

Manage database logs

Customize collections

Advanced configuration

1

2

3

4

5

4

7

6

8

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Why TFA?

Provides one interface for all diagnostic needs

Collects data across the cluster and consolidates it in one place

Collects all relevant diagnostic data at the time of the problem

Reduces time required to obtain diagnostic data, which saves your business money

5

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• All major Operating Systems are supported – Linux (OEL, RedHat, SUSE, Itanium &

zLinux)

– Oracle Solaris (SPARC & x86-64)

– AIX

– HPUX (Itanium & PA-RISC)

– Windows (64bit)

• All Oracle Database & Grid versions 10.2+ are supported

• You probably already have TFA installed as it is included with:

• Updated quarterly via 1513912.1

6

Supported Platforms and Versions

Oracle Grid Infrastructure

Oracle Database

11.2.0.4+ 12.2.0.1+ 12.1.0.2+

12.2.0.1+

OS versions supported are the same as those supported by the Database Java Runtime Edition 1.8 required

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Root / Daemon Install

1. Download from 1513912.1

2. Copy to one required machine and unzip

3. Run

Will : – Install on all nodes

– Auto discover relevant Oracle Software & Exadata Storage Servers

– Start monitoring for problems & perform auto collections

Non root / Non Daemon Install

1. Download from 1513912.1

2. Copy to every required machine and unzip

3. Run Will:

– Only install on current host

– Not do automatic collections

– Not collect from remote hosts

– Not collect files unreadable by install user

7

Linux / Unix Installation

./installTFA<platform>

Recommended install location: /opt/oracle.tfa

./installTFA<platform>

-extractto <install_dir>

-javahome <jre_home>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Windows Installation 1. Download from 1513912.1

2. Copy to one required machine and unzip

3. Open a command prompt as administrator and run:

Will : – Prompt for local or cluster install and install remotely if required

– Auto discover relevant Oracle Software

– Start monitoring for problems & perform auto collections

8

install.bat –perlhome <perl_home>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Key TFA Directories

9

Depending on how TFA was installed the tfa_home and bin directory may be in a different location, e.g. when installed as part of Grid Infrastructure the tfa_home path will be under grid_home

Directory Description

tfa/bin Contains the command line interface tfactl ( tfactl is also in

GRID_HOME/bin directory if Oracle Clusterware is installed)

tfa/repository Where Oracle Trace File Analyzer stores collections

tfa/<node>/tfa_home/database Contains Berkeley database that stores data about the system

tfa/<node>/tfa_home/diag Tools for troubleshooting Oracle Trace File Analyzer

tfa/<node>/tfa_home/diagnostics_to_collect Place files in here to include them in the next collection, then have

them deleted afterwards

tfa/<node>/tfa_home/log Contains logs about Oracle Trace File Analyzer operation

tfa/<node>/tfa_home/resources Contains resource files, for example the log masking control file

tfa/<node>/tfa_home/output Contains extra meta data about the environment

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 10

TFA Daemon

Remote Node

n

TFA Daemon

tfactl

Initiator Node ( Where command originated)

Cluster

• TFA daemon runs on each cluster node

• Or single instance when no Grid Infrastructure is used

• Command line communication is via tfactl command

• TFA Daemons on all nodes coordinate: • Script execution • Collection of diagnostics • Trimming of log contents

• Cluster wide collection output is consolidated on one node

Architecture

Cluster wide

Collection

Alerts & Log files

Scripts

TFA Daemon

Remote Node

2

Scripts

TFA Daemon

Remote Node

1

Alerts & Log files Scripts

Scripts

The daemon is only used when installed as root

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Automatic Diagnostic Collections

11

Automatically detect event

Collect & package relevant

diagnostics

Notify relevant DBA and or Sys Admin by

email

Upload collection to Oracle Support for further help

Significant problem occurs

1

2

3

4

Oracle Trace File Analyzer DBA(s) / Sys Admin(s)

Oracle Grid Infrastructure & Database(s)

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

On-demand Analysis and Collection

12

Problem occurs

Collect & package relevant

diagnostics

Provide summary of systems &

health

2

Perform analysis using tools where

required

View system & cluster summary

1

Collect diagnostics

Investigate logs & analyze

Upload package to Oracle Support for help resolving

Resolve problem

Need more help

?

Need more info

? Oracle Grid Infrastructure & Database(s)

Oracle Trace File Analyzer DBA(s) / Sys Admin(s)

5 6

7

8 9 10

11

No

Yes

No

Yes

Any Problems

? Yes

3 No

4

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Specify all command options at the command line

1. Set and change context

2. Run commands from within the shell

13

1. Select menu navigation options then choose the command you want to run

Command Interfaces

Command line Shell Menu

tfactl <command>

tfactl

tfaclt > database MyDB

MyDB tfactl > oratop

tfactl menu

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Understand TFA & install

Basic configuration

View system & cluster summary

Investigate logs & analyze

Collect diagnostics

Manage database logs

Customize collections

Advanced configuration

1

2

3

4

5

14

7

6

8

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

mask_strings.xml • If you want to mask sensitive data – Hostname

– IP address

– Etc

• Create mask_strings.xml – In the directory tfa_home/resources

– Define all data replacements you want

15

Data Redaction

<mask_strings>

<mask_string>

<original>WidgetNode1</original>

<replacement>Node1</replacement>

</mask_string>

<mask_string>

<original>192.168.5.1</original>

<replacement>Node1-IP</replacement>

</mask_string>

<mask_string>

<original>WidgetNode2</original>

<replacement>Node2</replacement>

</mask_string>

<mask_string>

<original>192.168.5.2</original>

<replacement>Node2-IP</replacement>

</mask_string>

</mask_strings>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Enable or disable user access

Configure User Access

tfactl

Grid Home owner

Database Home Owners

Everyone Else

Users & groups authorized to access tfactl by default

tfactl access enable

tfactl access disable

tfactl access add –user <user> [-local]

tfactl access remove –user <user> [-local]

tfactl access removeall [-local]

• Add users

• Remove users or everyone

All commands apply cluster wide unless -local used to restrict to local node

• Reset to default access

tfactl access reset

• List user access

tfactl access lsusers

16

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Set Email Notification Addresses

17

tfactl set [email protected]

Automatically detect event

Collect & package relevant

diagnostics

Notify relevant DBA

and or Sys Admin by email

Upload collection to

Oracle Support for further help

Significant problem occurs

1

2

3

4

Oracle Trace File Analyzer DBA(s) / Sys Admin(s)

Oracle Grid Infrastructure & Database(s)

tfactl set notificationAddress=oracle:[email protected]

• TFA can send email notification when significant problems are detected

• To set notification email for any problem detected:

• To set notification email for specific ORACLE_HOMEs include the OS owner:

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Option 1 – Applying standard PSUs will

automatically update TFA

– PSUs do not contain Support Tools Bundle updates

• Option 2 – To update with latest TFA & Support

Tools Bundle

1. Download latest version: 1513912.1

2. Repeat the same installation steps

18

Maintain

Upgrade to the latest version whenever possible to include bug fixes, new features & optimizations

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Understand TFA & install

Basic configuration

View system & cluster summary

Investigate logs & analyze

Collect diagnostics

Manage database logs

Customize collections

Advanced configuration

1

2

3

4

5

19

7

6

8

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

On-demand Analysis and Collection

20

Problem occurs

Collect & package relevant

diagnostics

Provide summary of systems &

health

2

Perform analysis using tools where

required

View system & cluster summary

1

Collect diagnostics

Investigate logs & analyze

Upload package to Oracle Support for help resolving

Resolve problem

Need more help

?

Need more info

? Oracle Grid Infrastructure & Database(s)

Oracle Trace File Analyzer DBA(s) / Sys Admin(s)

5 6

7

8 9 10

11

No

Yes

No

Yes

Any Problems

? Yes

3 No

4

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

View System & Cluster Summary

21

Quick summary of status of key components

Choose an option to drill down further

Summary is currently only available on Linux

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Summary ASM Drill Down Example

22

Problems found

ASM Cluster wide status

ASM Overview

ASM cluster wide summary

Problems found on myserver69

Also disk space warning on both servers

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Summary ASM Drill Down Example

23

View recent problems detected

View component status

View node wise & drill into myserver69

View ASM status summary for myserver69

View ASM problems for myserver69

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Understand TFA & install

Basic configuration

View system & cluster summary

Investigate logs & analyze

Collect diagnostics

Manage database logs

Customize collections

Advanced configuration

1

2

3

4

5

24

7

6

8

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Investigate Logs & Look for Errors

25

Problem occurs

Collect & package relevant

diagnostics

Provide summary of systems &

health

2

Perform analysis using tools where

required

View system & cluster summary

1

Collect diagnostics

Investigate logs & analyze

Upload package to Oracle Support for help resolving

Resolve problem

Need more help

?

Need more info

? Oracle Grid Infrastructure & Database(s)

Oracle Trace File Analyzer DBA(s) / Sys Admin(s)

5 6

7

8 9 10

11

No

Yes

No

Yes

Any Problems

? Yes

3 No

4

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Analyze all important recent log entries: • Search recent log entries:

26

Investigate Logs & Look for Errors

tfactl analyze –last 1d tfactl analyze -search “ora-006" -last 8h

Searching for “ora-00600”

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Perform Analysis Using the Included Tools

Not all tools are included in Grid or Database install. Download from 1513912.1 to get full collection of tools

Tool Description

orachk or exachk

Provides health checks for the Oracle stack. Oracle Trace File Analyzer will install either • Oracle EXAchk for Engineered Systems, see document 1070954.1 for

more details or • Oracle ORAchk for all non-Engineered Systems, see document

1268927.2 for more details

oswatcher Collects and archives OS metrics. These are useful for instance or node evictions & performance Issues. See document 301137.1 for more details

procwatcher Automates & captures database performance diagnostics and session level hang information. See document 459694.1 for more details

oratop Provides near real-time database monitoring. See document 1500864.1 for more details.

sqlt Captures SQL trace data useful for tuning. See document 215187.1 for more details.

alertsummary Provides summary of events for one or more database or ASM alert files from all nodes

ls Lists all files TFA knows about for a given file name pattern across all nodes

pstack Generate process stack for specified processes across all nodes

Tool Description

grep Search alert or trace files with a given database and file name pattern, for a search string.

summary Provides high level summary of the configuration

vi Opens alert or trace files for viewing a given database and file name pattern in the vi editor

tail Runs a tail on an alert or trace files for a given database and file name pattern

param Shows all database and OS parameters that match a specified pattern

dbglevel Sets and unsets multiple CRS trace levels with one command

history Shows the shell history for the tfactl shell

changes Reports changes in the system setup over a given time period. This includes database parameters, OS parameters and patches applied

calog Reports major events from the Cluster Event log

events Reports warnings and errors seen in the logs

managelogs Shows disk space usage and purges ADR log and trace files

ps Finds processes

triage Summarize oswatcher/exawatcher data

27

Verify which tools you have installed: tfactl toolstatus

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 28

Oracle Stack Health Checks

• Oracle EXAchk for Engineered Systems

• Oracle ORAchk for everything else • Proactively scans for the most impactful problems across the layers of your stack

• Simplifies and streamlines how to investigate and analyze which known issues present a risk to you

• Lightweight tool that runs within your environment without requiring config data to be sent to Oracle

• High level reports show your system health risks with the ability to drill down into specific problems and understand their resolutions

• Can be configured to send email notifications when it detects problems

• Collection Manager, a companion Application Express web app, provides a single dashboard view of collections across your entire enterprise

ORAchk & EXAchk (Support Tools Bundle)

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 29

OS Watcher (Support Tools Bundle)

Collect & Archive OS Metrics • Executes standard UNIX utilities (e.g. vmstat, iostat, ps,

etc) on regular intervals

• Built in Analyzer functionality to summarize, graph and report upon collected metrics

• Output is Required for node reboot and performance issues

• Simple to install, extremely lightweight

• Runs on ALL platforms (Except Windows)

• MOS Note: 301137.1 – OS Watcher Users Guide

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 30

Procwatcher (Support Tools Bundle)

Monitor & Examine Database Processes

• Single instance & RAC

• Generates session wait, lock and latch reports as well as call stacks from any problem process(s)

• Ability to collect stack traces of specific processes using Oracle Tools and OS Debuggers

• Typically reduces SR resolution for performance related issues

• Runs on ALL major UNIX Platforms

• MOS Note: 459694.1 – Procwatcher Install Guide

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 31

oratop (Support Tools Bundle)

Near Real-Time Database Monitoring

• Single instance & RAC

• Monitoring current database activities

• Database performance

• Identifying contentions and bottleneck

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Analyze

• Each tool can be run using tfactl in shell mode

• Start tfactl shell with

• Run a tool with the tool name

1. Where necessary set context with database <dbname>

2. Then run tool

3. Clear context with database

32

tfactl

tfactl > database MyDB

MyDB tfactl > database

tfactl > orachk

MyDB tfactl > oratop

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Understand TFA & install

Basic configuration

View system & cluster summary

Investigate logs & analyze

Collect diagnostics

Manage database logs

Customize collections

Advanced configuration

1

2

3

4

5

33

7

6

8

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Collect Diagnostics

34

Problem occurs

Collect & package relevant

diagnostics

Provide summary of systems &

health

2

Perform analysis using tools where

required

View system & cluster summary

1

Collect diagnostics

Investigate logs & analyze

Upload package to Oracle Support for help resolving

Resolve problem

Need more help

?

Need more info

? Oracle Grid Infrastructure & Database(s)

Oracle Trace File Analyzer DBA(s) / Sys Admin(s)

5 6

7

8 9 10

11

No

Yes

No

Yes

Any Problems

? Yes

3 No

4

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• For certain types of problems Oracle Support will ask you to run a Service Request Data Collection (SRDC)

• Previously this would have involved:

• Reading many different support documents

• Collecting output from many different tasks

• Gathering lots of different diagnostics

• Packaging & uploading

• Now just run:

One Command SRDCs

tfactl diagcollect -srdc <srdc_type>

35

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 36

Faster & Easier SR Data Collection

Type of Problem SRDC Types Collection Scope

ORA Errors

• ORA-00600

• ORA-00700

• ORA-04030

• ORA-04031

• ORA-07445

• ORA-27300

• ORA-27301

• ORA-27302

Local only

Other internal database errors • internalerror Local only

Database performance problems • dbperf Cluster wide

Database patching problems • dbpatchinstall

• dbpatchconflict Local only

Database install / upgrade problems • dbinstall

• dbupgrade Local only

Enterprise Manager tablespace usage metric problems • emtbsmetrics Local only (on EM Agent target)

Enterprise Manager general metrics page or threshold

problems - Run all three SRDCs

• emdebugon

• emdebugoff Local only (on EM Agent target & OMS)

• emmetricalert Local only (on EM Agent target & Repository DB)

New

New

New

New

New

New

New

New

tfactl diagcollect –srdc <srdc_type>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

ORA4031:

1. IPS Package

2. Patch Listing

3. AWR report

4. Memory information

5. RDA

Database Performance

1. ADDM report

2. AWR for good and problem period

3. AWR Compare Period report

4. ASH report for good and problem period

5. OS Watcher

6. IPS Package (if errors during problem period)

7. ORAchk (performance related checks)

One Command SRDCs – Examples of What’s Collected

tfactl diagcollect –srdc ora4031 tfactl diagcollect –srdc dbperf

37

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Manual Data Gathering vs One Command SRDC

Manual Data Gathering

1. Generate ADDM reviewing Document 1680075.1

2. Identify “good” and “problem” periods and gather AWR reviewing Document 1903158.1

3. Generate AWR compare report (awrddrpt.sql) using “good” and “problem” periods

4. Generate ASH report for “good” and “problem” periods reviewing Document 1903145.1

5. Collect OSWatcher data reviewing Document 301137.1

6. Check alert.log if there are any errors during the “problem” period

7. Find any trace files generated during the “problem” period

8. Collate and upload all the above files/outputs to SR

TFA SRDC

1. Run

2. Upload resulting zip file to SR

tfactl diagcollect –srdc dbperf

38

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

1. Enter default for event date/time and database name

2. Scans system to identify recent 10 events in the system (ORA600 example shown)

3. Once the relevant event is chosen, proceeds with diagnostic collection

39

One Command SRDC

tfactl diagcollect –srdc <srdc_type>

4. All required files are identified

5. Trimmed where applicable

6. Package in a zip ready to provide to support

Interactive Mode

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

One Command SRDC

40

tfactl diagcollect –srdc <srdc_type> -database <db> -for <time>

1. Parameters(date/time, DB name) are provided in the command

2. Does not prompt for any more information

3. All required files are identified

4. Trimmed where applicable

5. Package in a zip ready to provide to support

Silent Mode

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Run a default diagnostic collection if there is not yet an SRDC about your problem:

• Will trim & collect all important log files updated in the past 12 hours:

• Collections stored in the repository directory

• Change diagcollect timeframe with

–last <n>h|d

41

Default Collection

tfactl diagcollect

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Understand TFA & install

Basic configuration

View system & cluster summary

Investigate logs & analyze

Collect diagnostics

Manage database logs

Customize collections

Advanced configuration

1

2

3

4

5

42

7

6

8

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Automatic Database Log Purge

• TFA can automatically purge database logs – OFF by default

– Except on a Domain Service Cluster (DSC), which it is ON by default

• Turn auto purging on or off:

• Will remove logs older than 30 days – configurable with:

• Purging runs every 60 minutes – configurable with:

43

tfactl set manageLogsAutoPurge=<ON|OFF>

tfactl set manageLogsAutoPurgePolicyAge=<n><d|h>

tfactl set manageLogsAutoPurgeInterval=<minutes>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Manual Database Log Purge

• TFA can manage ADR log and trace files

– Show disk space usage of individual diagnostic destinations

– Purge these file types based on diagnostic location and or age:

• "ALERT“, "INCIDENT“, "TRACE“, "CDUMP“, "HM“, "UTSCDMP“, "LOG“ tfactl managelogs <options>

Runs as the ADR home owner. So will only be able to purge files this owner has permission to delete

Option Description

–show usage Shows disk space usage per diagnostic directory for both GI and database logs

-show variation –older <n><m|h|d> Use to determine per directory disk space growth.

Shows the disk usage variation for the specified period per directory.

-purge –older <n><m|h|d> Remove all ADR files under the GI_BASE directory, which are older than the time specified

–gi Restrict command to only diagnostic files under the GI_BASE

–database [all | dbname] Restrict command to only diagnostic files under the database directory. Defaults to all,

alternatively specify a database name

-dryrun Use with –purge to estimate how many files will be affected and how much disk space will be

freed by a potential purge command.

May take a while for a large number of files

44

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 45

Manual Database Log Purge

tfactl managelogs –show usage tfactl managelogs –show variation –older <n><m|h|d>

Use -gi to only show grid infrastructure

Use –database to only show database

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 46

Manual Database Log Purge tfactl managelogs –purge –older n<m|h|d> -dryrun tfactl managelogs –purge –older n<m|h|d>

Use –dryrun for a “what if”

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Disk Usage Snapshots

• TFA will track disk usage and record snapshots to: – tfa/repository/suptools/<node>/managelogs/usage_snapshot/

• Snapshot happens every 60 minutes, configurable with:

• Disk usage monitoring is ON by default, configurable with:

47

tfactl set diskUsageMonInterval=<minutes>

tfactl set diskUsageMon=<ON|OFF>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Change Trace Levels

• Sometimes Oracle Support may require certain specific trace levels to be enabled when reproducing a problem

• TFA makes it easy to enable and disable the correct trace levels with a single command

• All the required trace level settings are organised into problem specific profiles

• The relevant profile trace logging can be enabled with:

48

tfactl dbglevel –set <profile>

• Profiles only available for Clusterware today, Database profiles coming soon..

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Understand TFA & install

Basic configuration

View system & cluster summary

Investigate logs & analyze

Collect diagnostics

Manage database logs

Customize collections

Advanced configuration

1

2

3

4

5

49

7

6

8

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 50

Time

Custom Collection Options

Option Description

-last <n><h|d> Collect since the previous n hours or

days

-from “<yyyy-mm-dd>” Collect from the date and optionally

time specified.

Valid date / time formats:

"Mon/dd/yyyy hh:mm:ss"

or "yyyy-mm-dd hh:mm:ss"

or "yyyy-mm-ddThh:mm:ss"

or "yyyy-mm-dd "

Option Description

–from "yyyy-mm-dd"

-to "yyyy-mm-dd"

Collect to the date and optionally time

specified.

Valid date / time formats:

"Mon/dd/yyyy hh:mm:ss"

or "yyyy-mm-dd hh:mm:ss"

or "yyyy-mm-ddThh:mm:ss"

or "yyyy-mm-dd"

-for “yyyy-mm-dd” Collect for the specified date.

Valid date formats:

"Mon/dd/yyyy"

or "yyyy-mm-dd"

tfactl diagcollect <option>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 51

Nodes, Databases & Components

Custom Collection Options

Option Description

-node Limit collection to comma separated list of nodes

-database Limit collection to comma separated list of databases

-asm Collect ASM logs

-crsclient Collect Client Logs that are under GIBASE/diag/clients

-dbclient Collect Client Logs that are under DB ORABASE/diag/clients

-dbwlm Collect DBWLM logs

-tns Collect TNS logs

-rhp Collect RHP logs

-procinfo Collect Gathers stack and fd from /proc for all processes

-afd Collect AFD logs

-crs Collect CRS logs

-wls Collect WLS logs

Option Description

-emagent Collect EMAGENT logs

-oms Collect OMS logs

-ocm Collect OCM logs

-emplugins Collect EMPLUGINS logs

-em Collect EM logs

-acfs Collect ACFS logs and Data

-install Collect Oracle Installation related files

-cfgtools Collect CFGTOOLS logs

-os Collect OS files such as /var/log/messages

-ashhtml Collect Generate ASH HTML Report

-ashtext Collect Generate ASH TEXT Report

-awrhtml Collect AWRHTML logs (requires –database option as well)

tfactl diagcollect <option>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 52

Misc

Custom Collection Options

Option Description

-collectdir Collects other files in specified comma separated directory list. Only files updated in the relevant time

range (12 hours by default) get collected.

-tag Store the collection in a use a custom directory name within the repository, using the tag value specified

-z Append the value specified to the collection zip name

-nocopy Prevent the copying of zip files back to the initiating node

-notrim Prevent the trimming of files

-silent Start collection as a background process so output is not echoed to the console

-nocores Do not collect core files

tfactl diagcollect <option>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• TFA can invoke IPS to query and collect these packages

Simple IPS (Incident Packaging Service) interface

• Database incidents are stored in the ADR (Automatic Diagnostic Repository)

• IPS packages details about these incidents for later diagnosis

53

Command Description

tfactl ips Invoke IPS

tfactl ips show incidents Show all IPS incidents

tfactl ips show problems Show all IPS problems

tfactl ips show package Show all IPS packages

tfactl diagcollect -ips –help Show all available diagcollect IPS options

tfactl diagcollect –ips Perform an IPS collection following prompts

tfactl diagcollect -ips -adrbasepath

<adr_base> -adrhomepath <adr_home>

Perform an IPS collection in silent mode.

tfactl diagcollect -ips -incident <incident id> Collect ADR details about a specific incident id

tfactl diagcollect -ips -problem <problem id> collect ADR details about a specific problem id

tfactl ips <options>

tfactl diagcollect –ips <options>

or

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Display details of incidents recorded in ADR

54

Show Incidents

tfactl ips show incidents

• Incident ID

• Problem Key

• Create Time

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Display details of problems recorded in ADR

55

Show Problems

tfactl ips show problems

• Problem ID

• Problem Key

• Last Incident

• Last Incident Time

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Display details of packages recorded in ADR

56

Show Package

tfactl ips show package

List of packages &

details of each package

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 57

Collect Packages

tfactl diagcollect -ips

Select ADR home

Generate package

Rest of collection

Resulting collection zips

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Understand TFA & install

Basic configuration

View system & cluster summary

Investigate logs & analyze

Collect diagnostics

Manage database logs

Customize collections

Advanced configuration

1

2

3

4

5

58

7

6

8

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 59

Query Status & Configuration

Option Description

status Show status of Oracle Trace File Analyzer clusterwide

config Print the current configuration details

directories List all the directories which Oracle Trace File Analyzer collects from .

hosts Lists the hosts registered with Oracle Trace File Analyzer. Only these can receive clusterwide

commands.

actions Lists all the actions submitted. For example diagnostic collections. By default, will only show

actions running or completed in the past hour.

repository Prints the current location and amount of used space of the repository directory.

status Show status of Oracle Trace File Analyzer clusterwide

tfactl print <option>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 60

Configuration Listings & Descriptions Configuration Listing Default Value Description

Automatic diagnostic collection ON Auto collect if a significant problem occurs. Possible values: ON, OFF

Trimming of files during diagcollection ON Trim log files to only entries within the time range of the collection.

Possible values: ON,OFF

Repository maximum size in MB

Smaller of either 10GB

or 50% of free space in

the file system

The largest size the repository can be.

Trace Level 1

Increases the level of verbosity. Possible values are: 1 / 2 / 3/ 4

A value of 1 results in the least amount of trace. A value of 4 results in

the most amount of trace.

You should only change this value at the request of Oracle Support

Automatic Purging ON

Will purge collections when: Free space in the repository falls below

1GB or Before closing the repository. Purging removes collections from

largest size through to smallest. This continues until the repository has

enough space to open.

Minimum Age of Collections to Purge (Hours) 12 The least number of hours to keep a collection, after which it is eligible

for purging.

Minimum Space free to enable Alert Log Scan (MB) 500 Will suspend log scanning if free space in the tfa_home falls below this

value.

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Start & Stop

• Start the daemon:

• Stop the daemon:

Restarting

• Enable auto restart of the daemon:

• Disable auto restart of the daemon:

61

Daemon Lifecycle

tfactl start

tfactl stop

tfactl enable

tfactl disable

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Repository will close and collections will stop if any of the following occur:

– Free Space in TFA_HOME is less than 100 MB

• Will also stop indexing

– Free Space in ORACLE_BASE is less than 100 MB

• Will also stop indexing

– Free Space in Repository is less than 1 GB

– Repository current size is greater than the repository max size (reposizeMB)

• The daemon will purge the repository when either: – Free space falls below 1GB

– Before closing the repository

62

Manage the Repository & Collections

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Show the status of the TFA repository

• Show statistics about collections

• Set the largest size of the repository

• Change the location of the repository

• Set the minimum age of collections, before they are eligible for purging.

• Turn on or off automatic purging of the repository

• Manually purge all collections older than the specified time

63

Repository Commands

tfactl print repository

tfactl print collections

tfactl set reposizeMB=<value>

tfactl set repositorydir=<dir>

tfactl set minagetopurge=<number of hours>

tfactl set autopurge=<ON|OFF>

tfactl purge -older x[h|d] [-force]

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Show current collection directories:

• Add directories:

• Change how a directory gets collected:

• Stop collecting from a directory:

Parameter Description

-private Do no collect files if the user executing does

not have read permission

-public Allow collection of files even if user does not

have read permission

-exclusions Only collect if the files match the restrictions

of: Type, Name & Time range

-noexclusions Only collect if the files match the restrictions

of: Time range

-collectall

Collect all files, without restriction. But only

for diagcollect –collectalldirs

When –collectalldirs is not used, these normal

restrictions apply: Type, Name & Time range

-node all | n1,n2... Apply change to either all cluster nodes, or

specific ones listed

64

Change Which Directories Get Collected

tfactl print directories <options>

tfactl directory add <dir> <options>

tfactl directory modify <dir> <options>

tfactl directory remove <dir> <options>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 65

Reduce the Size of Collections

• Print current config:

• Set a config parameter:

Parameter Default Value Description

trimfiles ON

ON : Trim files to only include data around the time of

the event

OFF: Do not trim files. Collect all files written to at the

time of the event

maxcorefilesize 20MB Set the largest size of Core File to collect. Ignore any

core files larger than this value.

maxcorecollectionsize 200MB

Set the largest size of collections containing core files. If

collection size reaches this value, do not collect any

more cores.

tfactl print config

tfactl set <parameter>=<value>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• View current hosts TFA knows about:

• Add a new host to the configuration for the first time:

• Add a host where the certificates are already synchronized

• Remove a host

66

Add or Remove a Host

tfactl print hosts

tfactl syncnodes

tfactl host add <host>

tfactl host remove <host>

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Change SSL Options

• Daemons use SSL/TLS on ports 5000 to 5005

• To change the ports:

1. Stop TFA on all nodes:

2. Edit tfa_home/internal/usableports.txt to replace the ports you want to use

3. Replicate the usableports.txt changes to all cluster nodes

4. Remove tfa_home/internal/port.txt from all cluster nodes

5. Start TFA on all nodes:

67

tfactl print hosts

tfactl start

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

68