adobe marketing cloud data workbench monitoring profile · 12. [optionally] do the same with the...
TRANSCRIPT
Adobe® Marketing Cloud
Data Workbench Monitoring Profile
Contents
Installing the Monitoring Profile.....................................................................................3
Workspaces for Monitoring the Data Workbench Server .............................................6Data Workbench Historic workspace ........................................................................................................................6
Data Workbench Profile Status workspace..............................................................................................................9
Data Workbench Server Status workspace............................................................................................................11
Data Workbench Profile Dimensions and Metrics.......................................................14Dimensions in the Data Workbench Historic profile..........................................................................................14
Dimensions in the Data Workbench Profile Status profile...............................................................................18
Dimensions in the Data Workbench Server Status profile ..............................................................................20
Metrics in the Data Workbench Historical Monitoring profile........................................................................23
Data Workbench Monitoring ProfileLast updated 10/16/2014
Installing the Monitoring ProfileDirections for installing the data workbench Monitoring Profile.
Installation Steps
Check the Installation Notes below for additional information when installing.
1. Configure a new Sensor instance as if it will be used for tagged web page data collection. Be sure the zig.gif file is in the Sensorweb server document root. Sensor can be run on the same host as the monitor profiles. (This is not an issue if using a textfile for this purpose.)
Note: This Sensor instance must be dedicated to receiving only traffic from the Monitoring Agents. Also, the Sensor canbe configured to run on a different port if you are re-purposing a web server for this collection.
2. In the txlogd.conf file there is the default line:
ContentFilterExclude image/,text/css,application/x-javascript,text/javascript
For the data workbench Monitoring Profile application (or any "tagged" page implementation), the image type has to beremoved in order to collect via a GIF file. The updated line is:
ContentFilterExclude text/css,application/x-javascript,text/javascript
3. Copy the insight_monitor.zip/insight_monitor_agent to a temporary location.4. Update insight_monitor_agent.cfg file for your environment. Follow the comments inside the configuration file:
The Monitoring configuration file:
Define where you are collecting all information and provide URL address. This needs to be a dedicated sensor and shouldreceive no traffic except for this application.
There are paths assuming there is an e: disk. You may want to change this path for your environment.
Sometime when running a Transform profile, data workbench can be unresponsive. This value lets you send an alert if threetimes in a row the process is unresponsive. This is a way of reducing false positive alerts.
This is where you set the environment and group dimensions. This may be different from host to host.
This is where you can see exactly what the monitor agent is doing by viewing error logs in this path.
3Installing the Monitoring Profile
This is to use the temp db internally. It may be alerted when reaching capacity. This is different than physical disk usage.
5. Copy the insight_monitor_agent folder to each DPU and FSU host running the data workbench server. The default locationas indicated in the configuration file is e:\insight_monitor_agent but you can change this location.
6. Add a Windows scheduled task to invoke the agent every 10 minutes (this period is assumed in the processing rate calculations).The program is e:insight_monitor/insight_monitor_agent.exe. The argument is config-filee:\insight_monitor\insight_monitor.cfg. Start in e:\insight_monitor. The user running executing the task must have permissionto read/write e:\insight_monitor and read the Win32 OLE object root\CIMV2 (required to ascertain the dataworkbench server service start mode and to check the percentage of space on local disks)
7. Confirm that the VSL file is starting to grow as monitor records accumulate. This will take some time as the traffic volumewill be extremely low in a small installation (every 10 minutes the agent sends only one hit for the host-specific data, plusone hit per processing profile).
8. Unzip insight_monitor.zip\profiles\Insight Historic to a temporary location.9. Update host name in profile.cfg, dataset\cluster.cfg, and the dataset\segment export.cfg.10. Update the files to the data workbench profiles directory.11. Update log server and path in dataset\log processing.cfg to the location where the Sensor VSLs are accumulating.12. [Optionally] do the same with the profiles Insight Profile Status and Insight Server Status. In addition, the status profiles
should be reprocessed nightly with a trailing two day window. Add a Windows scheduled task: The program ise:\insight_monitor\insight_reprocess.exe. The argument is --profile-path="PATH TOPROFILES\insight profile status" --start-days-ago=2. Leave start in blank. Add another scheduledtask for "insight server status". insight_reprocess.exe requires read/write access to log processing.cfg to update the start time.
13. In addition, the status profiles should be reprocessed nightly with a trailing two day window. Add a Windows scheduledtask: The program is e:\insight_monitor\insight_reprocess.exe. The argument is --profile-path="PATH TOPROFILES\insight profile status" --start-days-ago=2. Leave start in blank. Add another scheduledtask for "insight server status". insight_reprocess.exe requires read/write access to log processing.cfgto update the start time. Confirm that each profile is reading the monitor VSLs as they accumulate. Again, this will take sometime—probably hours—because of the extremely low volume.
Installation Notes
• Configuring the Monitoring Profile in a licensed test environment. The test environment package is included with yourimplementation of data workbench, allowing you to install and configure the application. If installing on a production FSUor DPU server, then you will need to configure the server to run on a separate port.
• Deploying a new Sensor specifically for the Monitoring Profile. You will need to install a new instance of Sensor to the serverrunning the Monitoring Profile. This is in addition to the production instance of Sensor. (There is no additional charge forinstalling Sensor on either a production or non-production server specifically for the Monitoring Profile.)
• Disable the monitor agent during data workbench maintenance. To avoid polluting the uptime and performance metrics,you can set service start mode to manual for the service InsightServer (Omniture Insight Server). A handy PowerShell commandis set-service -name insightserver -startuptype manual. Set it back to automatic after the maintenance: set-service -nameinsightserver -startuptype automatic. Another option is to temporarily disable the monitor agent scheduled task.
• The Status profiles need a trailing window to drop old hosts and profiles as well as old host-profile mappings. However, ifthe amount of event data is so small that data workbench won't buffer it in, then you may need to extend the size of the windowquite a bit to get it to process.
• The agent collects the overall and oldest as-of time from data workbench detailed status, which is reported in local hosttime assuming the event data log time stamps are in UTC (as in VSL files). If the event data timestamps are in a non-UTC
4Installing the Monitoring Profile
time zone the as-of time will be offset in the resulting Insight Profile Status profile. If all of your event data time stamps are inthe same time zone you can add that offset to Insight Profile Status\metrics\as of delay minutes.metric.
• Two new dimensions were introduced to help the customer group their servers if they are in different states, such asproduction, staging, testing servers, and servers in other states. For example if you are looking for "uptime", then you look atservers only in production mode. As a result, the Group dimension is just another way of arbitrarily grouping servers for yourneeds. For example, in theMonitoring Configuration file you can set which host your department is servicing, such as Operations,Development, or Marketing.
5Installing the Monitoring Profile
Workspaces for Monitoring the Data Workbench ServerTo successfully identify server health and performance, you can use standard data workbench profiles to monitor the serverfrom the installed agent using current data, or employ profiles of historic data sets to view the impact of performance changesover time.
The data workbench workspaces most commonly used include:
• Insight Historic workspace• Insight Profile Status workspace• Insight Server Status workspace
To select a profile, open the drop-down menu from the upper-left corner of the data workbench Client interface.
Data Workbench Historic workspace
Use the data workbench Historic profile to see how configuration, hardware, and other changes impact performance, stability,and server capacity over time.
The Historic profile includes a profile-based Profile Performance dataset and the server-based Server Performance dataset underthe Performance tab. These are the most commonly used datasets viewed for a past perspective of the data workbench serverperformance. In addition, you can view the Components and Processing Mode by selecting the Up Time tab.
In addition, you can view the Components and Processing Mode by selecting the Up Time tab.
6Workspaces for Monitoring the Data WorkbenchServer
For additional reference information about the dimensions used in the data workbench Historic profile, see Dimensions in theInsight Historical profile.
Profile Performance workspace
This dataset includes the following relevant metrics for data workbench monitoring.
• Fast Input MegaBytes per Minute—metrics displaying heavy data input during initial log processing.• Fast Merge MegaBytes per Minute—metrics displaying transformation.
Note: To do a real performance assessment of your profile, look at rate rather than elapsed calendar time. The rate ismeasured as the changed values between polling every ten minutes.
Server Performance workspace
This dataset monitors server metrics beyond the scope of included profiles, and includes the following relevant server metricsfor data workbench monitoring.
• Estimated Sweep Minutes — Estimated query resolution time.• Poll Latency Milliseconds — Indicator of how busy software is by measuring how long it takes to get through a full cycle of
servicing every component.
7Workspaces for Monitoring the Data WorkbenchServer
Components workspace
This dataset is located under the Up Time tab.
The Components dataset includes two aspects for component health:
• Communications metric — Did the data workbench server process respond?• All Components metric — At top of Detailed Status page is a list of components the host is servicing within the data workbench
server processes. If any component is in an error state then it is listed under the Components in Error table.
8Workspaces for Monitoring the Data WorkbenchServer
Processing Mode workspace
This workspace is located under the Up Time tab. This workspace lets you observe how much time is taken in fast input, fastmerge, and real-time modes.
This dataset provides important server load characteristics, such as identifying data load for
• Day of the week (for example a Fast Input Rate on Tuesday and Wednesday),• Hour of Day (what percentage of the day is it in Fast Input mode?)
Data Workbench Profile Status workspace
The data workbench Profile Status profile provides current information about the data workbench server health based on theprofile rather than server metrics or historical data.
9Workspaces for Monitoring the Data WorkbenchServer
Data Workbench Profile Status
This status profile provides the data workbench server information that is current, but not quite real-time because the agent ispolled every ten minutes and reporting always includes this ten-minute latency. More precisely, the datasets generated by thisprofile provide the latest observation of the server from the agent, which most often has a default polling period of ten minutes.
For additional reference information about the dimensions used in the data workbench Profile Status profile, see Insight ProfileStatus profile.
This report is more for monitoring operations rather than components or specific traffic fluctuations.
10Workspaces for Monitoring the Data WorkbenchServer
This gives us a sense of who is in what mode: If we see a high Fast Input rate for a certain profile then that profile is in Fast Inputmode.
If the Stalled metric is 1, then the server is stalled. If the value is 0, then the server is not stalled.
Log Reading for large batch loads
Data Workbench Server Status workspace
The data workbench Server Status profile provides current information about data workbench server health based on the serverrather than profile metrics or historical data.
11Workspaces for Monitoring the Data WorkbenchServer
General Status
Open the General Status dataset view within the data workbench Server Status profile.
For additional reference information about the dimensions used in the data workbench Server Status profile, see the Dimensionsin the Insight Server Status profile profile.
Disk Status
View current disk usage including internal usage of temp.db.
12Workspaces for Monitoring the Data WorkbenchServer
13Workspaces for Monitoring the Data WorkbenchServer
Data Workbench Profile Dimensions and MetricsThis document describes the profiles with their fields, dimensions, and metrics employed by the data workbench MonitoringProfile.
To monitor data workbench servers, data is collected using a script that parses the Detailed Status while also capturing specificserver information. Data workbench server information is then passed to a page tag call for the data workbench Sensor to collectand save to a VSL file.
Profiles Employed by the Data Workbench Monitoring Profile
These profiles provide dimensions and metrics that allow you to view server state and performance data:
• Dimensions in the Data Workbench Profile Status profile• Dimensions in the Data Workbench Server Status profile• Dimensions in the Data Workbench Historic profile• Metrics in the Data Workbench Historical Monitoring profile
The Status profiles allow you to see how data workbench is currently performing from an operational perspective. The ProfileStatus profile and the Server Status profile gather data from the Detailed Status and the data workbench servers. All gathereddata is placed into the cs-uri-query field for use.
The Historic profiles allow you to assess the impact of configuration and hardware changes using historical data. The historicalprofile may be the most useful because it allows you to assess the impact of configuration and hardware changes over time.
Dimensions in the Data Workbench Historic profile
The following dimensions are available for use in the data workbench Historic profile.
Dimensions in the Data Workbench Historic profile
The following dimensions are available for use in the data workbench Historic profile.
This is the only countable in this configuration, it is the root for all dimensions. A blockcan be considered a server. It is using the x-trackingid field.
Block
Note: The block ID is a hash of the server name plus host name, so there will beapproximately one block per server per profile.
This is a countable dimension built off of the Block countable. Each row of data in theprofile is a ping from the monitoring agent.
Ping
Numeric Dimension built from the cs-uri-query(ad) value. It is built at the Ping levelconditioned that cs-uri-query(a) matches "1".
Alert Critical
Numeric Dimension built from cs-uri-query(ac) value. It is built at the Ping level,conditioned that cs-uri-query(a) matches "1".
Alert Down
Numeric Dimension built from cs-uri-query(ae) value. It is built at the Ping levelconditioned that cs-uri-query(a) matches "1".
Alert Warning
14Data Workbench Profile Dimensions and Metrics
Numeric Dimension built from cs-uri-query(aa) value. It is built at the Ping levelconditioned that cs-uri-query(a) matches "1" and cs-uriquery(k) is not empty.
Any Profile Reprocessing
cs-uri-query(bi) is placed into the x-as-of-delay-minutes field and rounded to the nearestminute. It is built at the Ping level conditioned that cs-uri-query(a) matches "1".
As of Delay Minutes
Numeric Dimension built from cs-uri-query(r) value. It is built at the Ping levelconditioned that cs-uri-query(a) matches "1" and cs-uriquery(k) is not empty.
Capacity Row Percentage
Numeric Dimension built from cs-uri-query(n) value. It is built at the Ping levelconditioned that cs-uri-query(a) matches "1" and cs-uriquery(k) is not empty.
Capacity Size Percentage
Simple Dimension built from cs-uri-query(v) value. It is built at the Ping level, andconditioned that cs-uri-query(a) matches "1".
Component Check Success
cs-uri-query(ao) is split by the "|" delimiter and copied into the x-components-in-errorfield. Many to Many Dimension built from the x-components-in-error field. Built at thePing level.
Components in Error
Numeric Dimension built from cs-uri-query(l) value. It is built at the Ping levelconditioned that cs-uri-query(k) is not empty.
Detailed Check Seconds
Simple Dimension built from the cs-uri-query(k) value. It is built at the Ping levelconditioned that cs-uri-query(a) matches "1".
Detailed Check Success
cs-uri-query(bc) is copied into the x-dimension-gigabytes field. The x-dimension-gigabytesfield is user for this Simple Dimension, conditioned on cs-uri-query(a) matching "2".
Dimension GigaBytes
These Numeric Dimensions are configured from the cs-uri-query(ah, ai, aj, ak, al) values.They are built at the Ping level and conditioned on cs-uri-query(a) matches "1" andcs-uri-query(k) is not empty.
Disk "x" Used Percentage
The x-estimated-sweep-dekaseconds field is used in this Numeric Dimension. This is theestimated sweep time of the servers divided by ten (reduced resolution of sweepmeasurement to make dimension more reasonably sized).
Estimated Sweep Dekaseconds
Note: This dimension is hidden because it is only useful when averaged into a metric.
The cs-uri-query(bj) value is used for this dimension. The Last Row for a block is usedas the value for the dimension. If the dataset is in Fast Input this Numeric Dimension'svalue will display the MB per minute at which the system is inputting data.
Fast Input MegaBytes per Minute
Note: This dimension is hidden because it is only useful when averaged into a metric.
The cs-uri-query(bk) value is used for this dimension. The Last Row for a Block is usedas the value for the dimension. If the dataset is in Fast Merge This Numeric Dimension'svalue will display the MB per minute at which the system is merging.
Fast Merge MegaBytes per Minute
Note: This dimension is hidden because it is only useful when averaged into a metric.
15Data Workbench Profile Dimensions and Metrics
The cs-uri-query(bg) value is used for this dimension. The value is divided by 1000 androunded to the nearest whole number. This Numeric Dimension's value will display theamount of space the Fields in the dataset are using.
Field GigaBytes
Note: This dimension is hidden because it is only useful when averaged into a metric.
Simple Dimension built at the Ping level from the cs-uri-query(x) value.Group
The cs-uri-query(b) value is used for this dimension. The Simple dimension's value isthe Last Row for a Block.
Host
the x-unixtime field is copied into x-last-ping and divided by 10 to reduce the cardinality.The Numeric Dimension is built at the Block level and uses the x-last-ping field.
Last Ping
This is a Numeric dimension using the Last Row for a given Server's cs-uri-query(i) value.It is conditioned on cs-uri-query(k) not being empty. This dimension is used to calculatethe average load on the servers in the system being monitored.
Load Average
Note: This dimension is hidden because it is only useful when averaged into a metric.
the cs-uri-query(be) value is used for this numeric dimension, built at the Ping level. Thisdimension is used to calculate the percentage of logs being read.
Log Reading Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
This is a Numeric dimension using cs-uri-query(o) value, built at the Ping level. It isconditioned on cs-uri-query(k) not being empty, and cs-uri-query(a) matching "1". Thisdimension is used to calculate the percent of page file memory usage.
Memory Page File Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
This is a Numeric dimension using the cs-uri-query(ag) value, built at the Ping level. Itis conditioned on cs-uri-query(k) not being empty, and cs-uri-query(a) matching "1.
Memory Physical MegaBytes Total
Note: This dimension is hidden because it is only useful when averaged into a metric.
This is a Numeric dimension using the cs-uri-query(ag) value, built at the Ping level. Itis conditioned on cs-uri-query(k) not being empty, and cs-uri-query(a) matching "1. Thisdimension is used to calculate the percent of physical memory usage of each Server.
Memory Physical Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
16Data Workbench Profile Dimensions and Metrics
This is a Numeric dimension using the cs-uri-query(s) value at the Ping level. It isconditioned on cs-uri-query(k) not being empty and cs-uri-query(a) matching "1. Thisdimension is used to calculate the percent of query memory usage of each Server.
Memory Query Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
This is a Numeric dimension using the cs-uri-query(q) value built at the Ping level. It isconditioned on cs-uri-query(k) not being empty and cs-uri-query(a) matching "1. Thisis used to show the number of network connections there are for a given server.
Network Connections
Note: This dimension is hidden because it is only useful when averaged into a metric.
cs-uri-query(bh) value is divided by 100000 and copied into the x-output-rows field toreduce the size of the dimension. X-output-rows is used in a Numeric Dimension builtat the Ping level, conditioned that cs-uri-query(a) matches "2".
Output Rows
Simple Dimension built using the cs-uri-query(a) value at the Ping level. This shows whatkind of Ping was recorded.
Ping Type ID
The cs-uri-query(m) value is divided by 10 to reduce dimension size, and copied into thex-poll-latency-centiseconds field. This is a Numeric dimension built at the Ping level,
Poll Latency Centiseconds
conditioned that cs-uri-query(k) is not empty, and cs-uri-query(a) matches "1"/ Thisdimension is used to calculate the poll latency.
Note: This dimension is hidden because it is only useful when averaged into a metric.
The cs-uri-query(bb) value is used for this Simple Dimension, built at the Ping level. Itis conditioned that cs-uri-query(bb) is not empty, and that cs-uri-query(a) matches "2"
Processing Mode ID
Processing Mode ID allows one to see what mode of processing the system is in (FastInput, Fast Merge, Real Time).
Note: This dimension is hidden then re-exposed with friendly values in the client-sidedimension Processing Mode.
The cs-uri-query(ba) value is used for this Simple Dimension, it is built at the Ping level.This dimension displays the name(s) of the profile(s) currently being monitored.
Profile
The cs-uri-query(h) value is used for this Numeric Dimension. It is built at the Ping levelconditioned that cs-uri-query(a) matches "1".
Quick Check Seconds
This is a Simple dimension built from the cs-uri-query(g) value built at the Ping level. Itis used to calculate the quick check metric.
Quick Check Success
Numeric Dimension using the cs-uri-query(t) value built at the Ping level. It is conditionedthat cs-uri-query(a) matches "1".
Real Time Processing Percentage
Simple Dimension built from the cs-uri-query(bl) value built at the Ping level. Thisdimension displays the when the last contact with a data source occurred.
Source Furthest Behind
17Data Workbench Profile Dimensions and Metrics
Numeric Dimension built using the cs-uri-query(an) value, built at the Ping level. It isconditioned that cs-uri-query(k) is not empty, and cs-uri-query(a) matches "1". It is usedto calculate the percentage of used Temp DB space on a given server.
Temp DB Space Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
the cs-uri-query(bf) value is used for this numeric dimension. It is built at the Ping level.This dimension is used to calculate the percentage of complete data transformation.
Transformation Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
The cs-uri-query(ab) value is used for this Simple Dimension. It is built at the Ping leveland conditioned that cs-uri-query(ab) is not empty, and cs-uri-query(a) matches "1".This displays the version(s) of data workbench server running on each server.
Data Workbench Version
The cs-uri-query(ab) value is split and the major release value is copied into thex-insight-version-major field. It is a Simple Dimension built at the Ping level andconditioned that x-insight-version-major is not empty, and cs-uri-query(a) matches "1".
Data Workbench Version Major
Dimensions in the Data Workbench Profile Status profile
The following dimensions are available for use in the data workbench Profile Status profile.
This is the only countable in this configuration and exists as the root for all dimensions.A block can be considered a server. It is using the x-trackingid field. The block ID is a
Block
hash of the server name plus host name, so there will be approximately one block perserver per profile.
cs-uri-query(bi) is placed into the x-as-of-delay-minutes field and rounded to the nearestminute. As of Delay Minutes is a numeric dimension that takes the Last Row fromx-as-of-delay-minutes for a block.
As of Delay Minutes
The cs-uri-query(c) value is used for the Environment ID. The Last Row for a Block isused as the value for the dimension. This Simple Dimension will display the Environmentin which your Servers are running (provided it is configured properly).
This can be set in the insight_monitor_agent.cfg file
Environment
The cs-uri-query(bj) value is used for this dimension. The Last Row for a block is usedas the value for the dimension. If the dataset is in Fast Input this Numeric Dimension'svalue will display the MB per minute at which the system is inputting data.
Fast Input MegaBytes per Minute
Note: This dimension is hidden because it is only useful when averaged into a metric.
The cs-uri-query(bk) value is used for this dimension. The Last Row for a Block is usedas the value for the dimension. If the dataset is in Fast Merge This Numeric Dimension'svalue will display the MB per minute at which the system is merging.
Fast Merge MegaBytes per Minute
Note: This dimension is hidden because it is only useful when averaged into a metric.
18Data Workbench Profile Dimensions and Metrics
The cs-uri-query(bg) value is used for this dimension. The value is divided by 1000 androunded to the nearest whole number. This Numeric Dimension's value will display theamount of space the Fields in the dataset are using.
Field GigaBytes
Note: This dimension is hidden because it is only useful when averaged into a metric.
The cs-uri-query(b) value is used for this dimension. The Simple dimension's value is theLast Row for a Block.
Host
x-last-ping is x-unixtime divide by 10 (to accommodate Numeric dimensions sizeconstraints). Last Ping is the Last Row for a given Block, and it represents the last timethe monitoring agent logged the system health.
Last Ping
Note: This dimension is hidden because it is only useful when averaged into a metric.
the cs-uri-query(be) value is used for this numeric dimension. It is the Last Row for agiven Block. This dimension is used to calculate the percentage of logs being read.
Log Reading Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
The cs-uri-query(bb) value is used for this Simple Dimension. It the Last Row for a givenBlock. Processing Mode ID allows one to see what mode of processing the system is in(Fast Input, Fast Merge, Real Time).
Processing Mode ID
Note: This dimension is hidden then re-exposed with friendly values in the client-sidedimension Processing Mode.
The x-processing-stalled field is created through various conditions to indicate whetherthe profile is currently running or not. It is a simple dimension.
Processing Stalled
Note: This dimension works best when there are a large number of input logs tofairly distribute amongst the DPUs. If there is, for example, only one large file loadedper day, then data workbench can appear to "stall" for an hour or more resulting ina false positive reading from this dimension.
The cs-uri-query(ba) value is used for this Simple Dimension. This dimension displaysthe name(s) of the profiles currently being monitored.
Profile
The last row of cs-uri-query(bl) is copied into the x-source-furthest-behind field. TheSimple Dimension uses the Last Row for a given Block. This dimension displays the whenthe last contact with a data source occurred.
Source Furthest Behind
the cs-uri-query(bf) value is used for this numeric dimension. It is the Last Row for agiven Block. This dimension is used to calculate the percentage of complete datatransformation.
Transformation Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
Hour, Day, Week, Month, Hour of Day, and Day of Week are all derived from thex-timestamp field.
Time Dimensions
19Data Workbench Profile Dimensions and Metrics
Grouping word that gives you another way to filter the resulting dataset. Set in theinsight_monitor_agent.cfg file.
Group
The following lists the metrics included in the data workbench Profile Monitoring Profileand how they are derived.
Metrics
This metric is the sum of the As Of Delay Minutes for each Block, then divided by theBlocks metric.
As Of Delay Minutes
This metric is the sum of the As Of Delay Seconds for each Block, and divided by the totalnumber of Blocks. (As of Delay Seconds Dimension not configured out of the box)
As Of Delay Seconds
Sum one for each Block.Blocks
The sum of Fast Input MegaBytes per Minute for each Block divided by the number ofBlocks when Fast Input MegaBytes per Minute is greater than zero.
Fast Input MB per Minute
The sum of Fast Merge MegaBytes per Minute for each Block divided by the number ofBlocks when Fast Merge MegaBytes per Minute is greater than zero.
Fast Merge MB per Minute
The sum of Field Gigabytes for each Block divided by Blocks metric.Field GigaBytes
The As Of Time minus Last Ping Time.Last Ping Age
The sum of Last Ping for each Block divided by Blocks, then multiplied by 10.Last Ping Time
Where Log Reading Percentage is greater than zero, Log Reading is the sum of Log ReadingPercentage for each Block, divided by Blocks metric, all of which is divided by 10.
Log Reading
The sum of the Processing Stalled Dimension for each Block, divided by the Blocks metric.Processing Stalled
The sum of Transformation Percentage for each Block divided by the Blocks metric, whenTransformation Percentage is greater than zero, all divided by 10.
Transformation
Dimensions in the Data Workbench Server Status profile
The following dimensions are available for use in the data workbench Server Status profile.
Built from the x-trackingid field, this countable dimension represents the Servers currentlyrunning data workbench.
Server
The cs-uri-query(af) value is used for this Simple Dimension. It is the Last Nonblankvalue for a Server. This displays the build date and time for the version(s) of themonitoring agent running.
Agent Version
The cs-uri-query(aa) field is used for this Numeric Dimension, it is the value of the LastRow for a given Server, conditional on cs-uri-query(k) is not empty. This dimension isused to indicate if any profiles are Reprocessing.
Any Profile Reprocessing
The cs-uri-query(r) field is used for this Numeric Dimension, it is the value of the LastRow for a given Server, conditional on cs-uri-query(k) is not empty.
Capacity Row Percentage
The cs-uri-query(n) field is used for this Numeric Dimension, it is the value of the LastRow for a given Server, conditional on cs-uri-query(k) is not empty.
Capacity Size Percentage
The sc-ur-query(am) field is used for this Simple Dimension, it is the value of the LastNonblank value for a given Server. It displays the Common Name of the servers beingmonitored.
Common Name
20Data Workbench Profile Dimensions and Metrics
The cs-uri-query(v) field is used for this Simple Dimension, it is the value of the LastRow for a given Server. This dimension checks on the components of the server to verifythey are properly functioning.
Component Check Success
A Crossrows transformation takes the Last Row value of the cs-uri-query(ao) and copiesit into the x-components-in-error field. This Many to Many dimension displays anycomponents in error on servers being monitored.
Components in Error
The cs-uri-query(c) value is used for the Environment ID. The Last Row for a Block isused as the value for the dimension. This Simple Dimension will display the Environmentin which your Servers are running (provided it is configured properly).
Environment
Note: This dimension is set in insight_monitor_agent.cfg.
The x-estimated-sweep-dekaseconds field is used in this Numeric Dimension. This isthe estimated sweep time of the servers divided by ten (reduced resolution of sweepmeasurement to make dimension more reasonably sized).
Estimated Sweep Dekaseconds
Note: This dimension is hidden because it is only useful when averaged into a metric.
The cs-uri-query(b) value is used for this dimension. The Simple dimension's value isthe Last Row for a Block.
Host
x-last-ping is x-unixtime divide by 10 (to accommodate Numeric dimensions sizeconstraints). Last Ping is the Last Row for a given Block, and it represents the last timethe monitoring agent logged the system health.
Last Ping
This is a Numeric dimension using the Last Row for a given Server's cs-uri-query(i) value.It is conditioned on cs-uri-query(k) not being empty. This dimension is used to calculatethe average load on the servers in the system being monitored.
Load Average
Note: This dimension is hidden because it is only useful when averaged into a metric.
This is a Numeric dimension using the Last Row for a given Server's cs-uri-query(o)value. It is conditioned on cs-uri-query(k) not being empty. This dimension is used tocalculate the percent of page file memory usage.
Memory Page File Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
This is a Numeric dimension using the Last Row for a given Server's cs-uri-query(ag)value. It is conditioned on cs-uri-query(k) not being empty.
Memory Physical MegaBytes Total
Note: This dimension is hidden because it is only useful when averaged into a metric.
21Data Workbench Profile Dimensions and Metrics
This is a Numeric dimension using the Last Row for a given Server's cs-uri-query(ag)value. It is conditioned on cs-uri-query(k) not being empty. This dimension is used tocalculate the percent of physical memory usage of each Server.
Memory Physical Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
This is a Numeric dimension using the Last Row for a given Server's cs-uri-query(s)value. It is conditioned on cs-uri-query(k) not being empty. This dimension is used tocalculate the percent of query memory usage of each Server.
Memory Query Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
This is a Numeric dimension using the Last Row for a given Server's cs-uri-query(q)value. It is conditioned on cs-uri-query(k) not being empty. This is used to show thenumber of network connections there are for a given server.
Network Connections
Note: This dimension is hidden because it is only useful when averaged into a metric.
This dimension is used to calculate the poll latency. The cs-uri-query(m) value is dividedby 10 to reduce dimension size, and copied into the x-poll-latency-centiseconds field.This is a Numeric dimension which takes the Last Row for a given server.
Poll Latency Centiseconds
Note: This dimension is hidden because it is only useful when averaged into a metric.
This is a Simple dimension built from the cs-uri-query(g) value of the Last Row for agiven Server. It is used to calculate the quick check metric.
Quick Check Success
The last row of the cs-uri-query(an) value is copied into the x-temp-db-space-percentagefield. This is a Numeric Dimension that is used to calculate the percentage of used TempDB space on a given server.
Temp DB Space Percentage
Note: This dimension is hidden because it is only useful when averaged into a metric.
The cs-uri-query(ab) value is used for this Simple Dimension. It is the Last Nonblankvalue for a Server. This displays the version(s) of data workbench server running on eachserver.
Insight Version
Grouping word that gives you another way to filter the resulting dataset.Group
Note: This dimension is set in insight_monitor_agent.cfg.
The following lists the metrics included in the data workbench Profile Monitoring Profileand how they are derived.
Metrics
This is the Capacity Size metric times two plus the Capacity Row metric divided by 3.Capacity Overall
This is the sum of the Capacity Row Percentage for each Server divided by Servers metric.Capacity Row
22Data Workbench Profile Dimensions and Metrics
This is the sum of the Capacity Size Percentage for each Server divided by the Serversmetric.
Capacity Size
This is the number of Servers where Component Check Success equals one, divided bythe Server where Service is DPU or FSU, all multiplied by 100 (to make it a percentage).
Component Check
The Disk metrics are calculated by taking the sum of their Disk Used Percentage for eachServer, divided by the Servers metric.
Disk "x"
This is the sum of the Estimated Sweep Dekaseconds for each Server, divided by theServers metric where Estimated Sweep Dekaseconds is greater than zero, all divided by6.
Estimated Sweep Minutes
The sum of Last Ping for each Block divided by Blocks, then multiplied by 10.Last Ping Time
This is the sum of the Load Average for each Server, divided by the Servers metric.Load
This is the sum of the Memory Page File Percentage for each Server, divided by the Serversmetric.
Memory Page File
This is the sum of the Memory Physical Percentage for each Server, divided by the Serversmetric.
Memory Physical
This is the sum of the Memory Query Percentage for each Server, divided by the Serversmetric.
Memory Query
This is the sum of the Network Connections for each Server divided by the Servers metric.Network Connections
This is the sum of the Poll Latency Centiseconds for each Server, divided by the Serversmetric, all of which is multiplied by 10.
Poll Latency Milliseconds
This is the number of Servers where Quick Check Success equals one, divided by theServers metric, all of which is multiplied by 100.
Quick Check
This is the sum of one for each Server, or the total number of monitored Servers.Servers
This is the sum of the Temp DB Space Percentage for each Server, divided by the Serversmetric.
Temp DB
Metrics in the Data Workbench Historical Monitoring profile
The following lists the metrics included in the data workbench Historical Monitoring Profile and how they are derived.
The sum of the Alert Critical dimension for each Ping.Alert Criticals
The sum of the Alert Down dimension for each Ping.Alert Downs
The sum of the Alert Warning dimension for each Ping.Alert Warnings
The count of Pings where Component Check Success equals "1" divided by the Pingsmetric multiplied by 100.
All Components
This metric is the sum of the As Of Delay Minutes for each Ping, then divided by the Pingsmetric.
As Of Delay Minutes
23Data Workbench Profile Dimensions and Metrics
The sum of one for each Block.Blocks
All Blocks.Block All
The Capacity Size Metric times 2 plus the Capacity Row metric, divided by 3.Capacity Overall
The sum of the Capacity Row Percentage dimension for each Ping divided by the Pingsmetric.
Capacity Row
The sum of the Capacity Size Percentage dimension for each Ping, divided by the Pingsmetric.
Capacity Size
The number of Pings where Quick Check Success matches "1", divided by the Pings metric.Communications
The sum of the Detailed Check Seconds dimension for each Ping where the ping type is"server", divided by the Pings metric.
Detailed Check Seconds
The sum of Dimension Gigabytes for each Ping, divided by the Pings metric.Dimension GigaBytes
The Disk metrics are calculated by taking the sum of their Disk Used Percentage for eachPing, divided by the Pings metric.
Disk "x"
This is the sum of the Estimated Sweep Dekaseconds for each Ping, divided by the Pingsmetric where Estimated Sweep Dekaseconds is greater than zero, all divided by 6.
Estimated Sweep Minutes
The sum of Fast Input MegaBytes per Minute for each Ping divided by the number ofPings when Fast Input MegaBytes per Minute is greater than zero.
Fast Input MB per Minute
Pings where Processing Mode dimension is equal to "Fast input" divided by Pings.Fast Input Mode
The sum of Fast Merge Megabytes per Minute for each Ping, divided by Pings metric.Fast Merge MegaBytes per Minute
Pings where Processing Mode equals "fast merge" divided by the Pings metric.Fast Merge Mode
The sum of Field Gigabytes dimension for each Ping divided by Pings metric.Field GigaBytes
The sum of the Load Average dimension for each Ping, divided by the Pings metric.Load
The sum of Log Reading Processing dimension for each Ping, divided by the Pings metric,all divided by 10.
Log Reading
The sum of Memory Page File Percentage for each Ping, divided by the Pings metric.Memory Page
The sum of the Memory Physical Percentage dimension for each Ping, divided by thePings metric.
Memory Physical
The sum of the Memory Query Percentage for each Ping, divided by the Pings metric.Memory Query
The sum of Memory Physical MegaBytes Total dimension for each Ping, divided by thePings metric.
Memory Total GB
This is the sum of the Network Connections for each Ping divided by the Pings metric.Network Connections
The Pings metric multiplied by the Capacity Overall metric.Pings x Capacity Overall
The sum of the Poll Latency Centiseconds dimension for each Ping, divided by the Pingsmetric, all multiplied by 10.
Poll Latency Milliseconds
24Data Workbench Profile Dimensions and Metrics
The sum of one for each Ping where Estimated Sweep Dekaseconds is greater than "0",divided by the Pings metric where Ping Type equals "server".
Query Running
The sum of Quick Check Seconds for each Ping where Ping Type is equal to "server",divided by the Pings metric.
Quick Check Seconds
The sum of Output Rows dimension for each ping divided by the Pings metric, multipliedby 100000.
Output Rows
The number of Pings where Processing Mode dimension equals "real time", divided bythe Pings metric, all multiplied by 100.
Real Time Mode
100 minus the number of Pings where Processing Mode equals "real time" divided by thePings metric, multiplied by 100.
Reprocessing Mode
The sum of the Processing Stalled dimension in the Insight Profile Status profile.Stalled
The sum of Temp DB Space Percentage for each Ping, divided by the Pings metric.Temp DB
The sum of Transformation Percentage for each Ping divided by the Pings metric alldivided by 10.
Transformation
25Data Workbench Profile Dimensions and Metrics