autopilot insight user’s guide - nastel · autopilot insight user’s guide confidentiality...

AP-Insight 100.000

AutoPilot Insight User’s Guide CONFIDENTIALITY STATEMENT: THE INFORMATION WITHIN THIS MEDIA IS PROPRIETARY IN NATURE AND IS THE SOLE PROPERTY OF

NASTEL TECHNOLOGIES, INC. ALL PRODUCTS AND INFORMATION DEVELOPED BY NASTEL ARE INTENDED FOR LIMITED DISTRIBUTION

TO AUTHORIZED NASTEL EMPLOYEES, LICENSED CLIENTS, AND AUTHORIZED USERS. THIS INFORMATION (INCLUDING SOFTWARE,

ELECTRONIC AND PRINTED MEDIA) IS NOT TO BE COPIED OR DISTRIBUTED IN ANY FORM WITHOUT THE EXPRESSED WRITTEN PERMISSION

FROM NASTEL TECHNOLOGIES, INC.

© 2017 Nastel Technologies, Inc. All rights reserved.

AutoPilot Insight User Guide

AP-Insight 100.000 A © 2017 by Nastel Technologies, Inc.

PUBLISHED BY:

RESEARCH & DEVELOPMENT

NASTEL TECHNOLOGIES, INC.

88 SUNNYSIDE BLVD, SUITE 101

PLAINVIEW, NY 11803

COPYRIGHT © 2017. ALL RIGHTS RESERVED. NO PART OF THE CONTENTS OF THIS DOCUMENT MAY BE

PRODUCED OR TRANSMITTED IN ANY FORM, OR BY ANY MEANS WITHOUT THE WRITTEN PERMISSION OF

NASTEL TECHNOLOGIES.

DOCUMENT TITLE: AUTOPILOT INSIGHT USER’S GUIDE

DOCUMENT RELEASE DATE: AUGUST 2017

HISTORY OF THIS DOCUMENT

Release Date Document Number Summary

August 2017 AP-Insight 100.000 Initial Release


AP-Insight 100.000 i © 2017 by Nastel Technologies, Inc.

Contents

1. INTRODUCTION .............................................................................................................................. 1

1.1 Key Benefits .............................................................................................................................................. 1

1.2 Typical Use Cases ...................................................................................................................................... 2 1.2.1 Root Cause Analysis of Application Performance Problems ........................................................................ 2 1.2.2 Bayesian Classification and Probability ........................................................................................................ 3 1.2.3 Real User Monitoring ................................................................................................................................... 4 1.2.4 Managed File Transfer (MFT) ....................................................................................................................... 5 1.2.5 Application Performance Monitoring .......................................................................................................... 6 1.2.6 Mobile Analytics .......................................................................................................................................... 7 1.2.7 Kafka Monitoring ......................................................................................................................................... 8 1.2.8 Middleware Monitoring and Tracking ......................................................................................................... 9

1.3 Data .......................................................................................................................................................... 9

1.4 jKQL........................................................................................................................................................... 9

2 USING AUTOPILOT INSIGHT .................................................................................................... 10

2.1 Accessing Insight ..................................................................................................................................... 10

2.2 Landing Page ........................................................................................................................................... 10 2.2.1 Explore a Demo .......................................................................................................................................... 11

2.2.1.1 Order Tracking Demo ........................................................................................................................ 12 2.2.2 Analyze Your Data ...................................................................................................................................... 17

2.2.2.1 Finding Prior Imports ......................................................................................................................... 21 2.2.3 Go to Dashboard ........................................................................................................................................ 22

2.3 Sample Dashboards ................................................................................................................................. 23 2.3.1 Steps in the Order Process Business Milestone ......................................................................................... 24 2.3.2 Activity Scorecard Latest Week ................................................................................................................. 25 2.3.3 Geomap Events by Location ...................................................................................................................... 26 2.3.4 Orders for the Last Hour that Missed their SLA ......................................................................................... 27 2.3.5 Credit Validation Exceptions ...................................................................................................................... 27 2.3.6 Events for Latest Hour by Location ............................................................................................................ 28 2.3.7 Application Performance Index Analytics .................................................................................................. 29 2.3.8 SLA Violation Scorecard ............................................................................................................................. 30 2.3.9 Serious Event Distribution ......................................................................................................................... 30 2.3.10 Elapsed Time for Order Events .................................................................................................................. 31 2.3.11 Exponential Moving Average for ElapsedTIme .......................................................................................... 32 2.3.12 Anomaly Monitor ....................................................................................................................................... 33 2.3.13 Function Analysis ....................................................................................................................................... 33

2.4 Create a Dashboard ................................................................................................................................. 34 2.4.1 Main Menu ................................................................................................................................................ 36 2.4.2 Additional Functions .................................................................................................................................. 37

2.5 Create Viewlets ....................................................................................................................................... 37


AP-Insight 100.000 ii © 2017 by Nastel Technologies, Inc.

2.5.1 Enter a jKQL Query..................................................................................................................................... 37 2.5.2 Editing a Viewlet ........................................................................................................................................ 39 2.5.3 Edit Query .................................................................................................................................................. 40 2.5.4 Other Functions on the Edit Viewlet Menu ............................................................................................... 40 2.5.5 Import/Export Viewlets ............................................................................................................................. 41

2.6 Settings ................................................................................................................................................... 43 2.6.1 Date & Time ............................................................................................................................................... 43 2.6.2 Dashboard .................................................................................................................................................. 44 2.6.3 Repository .................................................................................................................................................. 45

3 FUNCTIONS .................................................................................................................................... 45

4 USING JKQL .................................................................................................................................... 46

4.1 Get .......................................................................................................................................................... 46

4.2 Show as ................................................................................................................................................... 47

Figures Figure 1. Root Cause Analysis ...................................................................................................................... 2

Figure 2. Bayesian Classification and Probaility ........................................................................................... 3

Figure 3. Real User Monitoring .................................................................................................................... 4

Figure 4. Managed File Transfers ................................................................................................................. 5

Figure 5. Application Performance Monitoring ........................................................................................... 6

Figure 6. Mobile Analytics ............................................................................................................................ 7

Figure 7. Kafka Monitoring ........................................................................................................................... 8

Figure 8. Middleware Monitoring and Tracking .......................................................................................... 9

Figure 9. Login Dialog Box .......................................................................................................................... 10

Figure 10. Landing Page ............................................................................................................................. 10

Figure 11. Settings Page ............................................................................................................................. 11

Figure 12. Start a Demo ............................................................................................................................. 12

Figure 13. Order Tracking Demo – Page 1 ................................................................................................. 13



Figure 16. Order Tracking Demo – ................................................................................................. 16

Figure 17. Upload File ................................................................................................................................ 17

Figure 18. Change Your File Selection ........................................................................................................ 17

Figure 19. Preview Uploaded File .............................................................................................................. 18

Figure 20. Advanced Options for Uploaded File ........................................................................................ 18

Figure 21. More Advanced Options for Uploaded File .............................................................................. 19

Figure 22. More Advanced Options for Uploaded File .............................................................................. 19


AP-Insight 100.000 iii © 2017 by Nastel Technologies, Inc.

Figure 23. Uploaded File Summary ............................................................................................................ 20

Figure 24. Viewlets for Uploaded File ........................................................................................................ 20

Figure 25. Add Uploaded File Viewlets to Dashboard ............................................................................... 21

Figure 26. List of Imports ........................................................................................................................... 21

Figure 27. Default Viewlets are Created .................................................................................................... 22

Figure 28. Publish Viewlets to a Dashboard............................................................................................... 22

Figure 29. Sample Dashboard .................................................................................................................... 23

Figure 30. More than One Dashboard ....................................................................................................... 23

Figure 31. Sample Viewlet – Steps in the Order Process Business Milestone ........................................... 24

Figure 32. Sample Viewlet – Activity Scorecard Latest Week .................................................................... 25

Figure 33. Sample Viewlet – Geomap Events by Location ......................................................................... 26

Figure 34. Sample Viewlet – Orders for the Last Hour that Missed their SLA ........................................... 27

Figure 35. Sample Viewlet – Credit Validation Exceptions ........................................................................ 27

Figure 36. Sample Viewlet – Events for Latest Hour by Location .............................................................. 28

Figure 37. Sample Viewlet – Application Performance Index Analytics .................................................... 29

Figure 38. Sample Viewlet – SLA Violation Scorecard................................................................................ 30

Figure 39. Sample Viewlet – Serious Event Distribution ............................................................................ 30

Figure 40. Sample Viewlet – Elapsed Time for Order Events ..................................................................... 31

Figure 41. Sample Viewlet – Exponential Moving Average for ElapsedTIme ............................................ 32

Figure 42. Sample Viewlet – Anomaly Monitor ......................................................................................... 33

Figure 43. Sample Viewlet – Function Analysis .......................................................................................... 33

Figure 44. Create New Dashboard Dialog Box ........................................................................................... 34

Figure 45. Default Viewlets ........................................................................................................................ 35

Figure 46. Main Menu ................................................................................................................................ 36

Figure 47. Create Viewlets Dialog Box ....................................................................................................... 37

Figure 48. Enter a jKQL Query .................................................................................................................... 37

Figure 49. Name Your Query ...................................................................................................................... 38

Figure 50. First Viewlet .............................................................................................................................. 38

Figure 51. Edit Viewlet Menu ..................................................................................................................... 39

Figure 52. Edit View Dialog Box ................................................................................................................. 39

Figure 53. Edit Query ................................................................................................................................. 40

Figure 54. Open Viewlet Dialog Box ........................................................................................................... 40

Figure 55. Viewlet Menu ............................................................................................................................ 41

Figure 56. Import Viewlet Dialog Box ........................................................................................................ 41

Figure 57. Export Viewlet Dialog Box ......................................................................................................... 42

Figure 58. Settings Dialog Box – Date & Time ............................................................................................ 43

Figure 59. Settings Dialog Box – Dashboard .............................................................................................. 44

Figure 60. Settings Dialog Box – Repository .............................................................................................. 45


AP-Insight 100.000 iv © 2017 by Nastel Technologies, Inc.

Tables Table 1. Main Menu Functions .................................................................................................................. 36


AP-Insight 100.000 1 © 2017 by Nastel Technologies, Inc.

1. Introduction For IT Ops professionals, members of the DevOps group, and developers, the ultimate advantage would

be to know everything as it happens in their business – and everything that could happen. To know all

and see all with complete vision is the competitive ideal: operations managers armed with real-time

analytics, detecting performance problems before delays arise; the company discovering trends the

moment they form.

If there is a function in modern technology that offers anything close to this ideal of omniscience, it’s

providing real-time analytics to prevent problems (or at least their impact) and forensics to resolve the

problems you can’t prevent.

Such awareness is difficult to attain, and it is often impossible for companies to know in advance what

events need to be analyzed and when that analysis must happen. IT must store and analyze everything, or

risk missing evidence of operational lags, risks, or rising customer trends. For security compliance

reasons alone, enterprises are required to maintain good logs, store them securely for at least one year,

and review them daily.

An important consideration is whether staff is looking at the data with the right degree of resolution. A

low-resolution view lets support staff isolate a problem to, say, a specific infrastructure tier and then pass

the problem to a specialist who will start diagnosing the problem all over again with a different tool.

While this common approach works, it is time-consuming, expensive, and disruptive. A more productive

approach would be to use a high-resolution analysis, enabling application support to diagnose a problem,

and with the forensics tools to immediately begin its resolution.

To answer business-centric questions and provide guidance for decision-makers, Nastel combines:

Analytics using advanced predictive anomaly detection and machine learning algorithms for

problem prevention across apps, messaging, logs, mobile, and the IoT

Insight into applications including: payment processing, trade compliance, order tracking,

healthcare claims processing, compliance, machine data, and more

Visibility across the IBM stack (MQ, IIB, DP, MFT), Java, mobile, and the newer open-source

technologies such as Kafka, STORM, Spark, MQTT, log files, Python, REST, and much more

Multi-tenancy with private data repositories available on premise or in SaaS

Lambda architecture with grids for real-time, in-memory analytics as well as historical analytics,

data replication, and time-to-live for all streaming data

End-to-end business transaction tracking that spans technologies, tiers, and organizations

Intuitive, easy-to-use data visualizations and dashboards.

All these capabilities fuse seamlessly across dynamic IT environments, from mobile to mainframe. They

provide the broad array of analytic and decision-support capabilities needed by developers, IT admins,

and business analysts to satisfy real-time operations intelligence and APM needs.

1.1 Key Benefits

Key benefits are insight, visibility, prediction, and machine learning that is easy-to-use to:

Improve service to customers and reduce operational risk – using AutoPilot’s machine

learning analytics

Highly scalable with self-service access, without need for data scientists – using flexible web-

based UI’s and natural language for ease of use and a powerful Lambda architecture with

microservices for scalability

Reduce support costs – via Docker deployment, open-source data collectors and ease of use.



1.2 Typical Use Cases

1.2.1 Root Cause Analysis of Application Performance Problems

Figure 1. Root Cause Analysis

Insight uses machine learning to detect anomalies in time-series data and can automatically determine the probable root

cause of this anomaly. It can create a dynamic visualization of application topology and show the chain of causality

between the anomaly and the applications that it has impacted. It can also detect if any business objectives or SLAs were

impacted by this anomaly.

The sample Viewlet above is using machine learning to detect anomalies. This scenario is based on real data representing

airport terminals and flights. We have represented an airline at a terminal as an application, a terminal as a server, a data

center as an airport, and the sky as a resource.



An anomaly was detected on February 18th with an average delay for the day of 45 minutes. If we click

on the anomaly, we are transported to the console for a drill-down showing the topology of that anomaly.

The graph shows a US Air flight traveling from Charlotte (CLT) to Phoenix (PHX). The red edges, called

rogue edges represent a problematic relationship between the terminal in Charlotte and the one in

Phoenix. Clicking on the rogue edge provides a root-cause analysis of the problem. There was a delay on

Charlotte and it took 8 times longer than average to get into the air. The average delay was about 9

minutes, while the worst actual delay was about 1 hour and 16 minutes.

While this example used airports, it’s easy to see how this would be applied to elapsed time for

applications in an IT operations use case.

1.2.2 Bayesian Classification and Probability

Figure 2. Bayesian Classification and Probaility

We use an algo based on Bayesian Classification or Naïve Bayes Theorem to automatically group or

classify entities and as a way to determine the probability of an event. This algo uses learning data and/or

learning queries and applies what it has learned to newly streamed data. Application of Bayes groups

data into sets and specifies a probability of confidence. Bayes is automatically run by the server and

applied to data as soon as it is streamed.

The scenario in the Viewlet above applies Bayes to the problem of “churn”. In this scenario we have

mobile users and we wish to predict which customers the carrier is at risk of losing.

The query in the upper Viewlet is a rule for learning how to determine and predict customers at risk and

customers that are satisfied. The lower Viewlet is showing the positive (61.54%) and negative (38.46%)

sentiment in the customer base.

Only classifications with greater than 75% probability (as seen in the jKQL above .75) are shown in the

chart above. Bayes is used in the sample dashboard called Sample-Mobile in the dashboard tab, Bayes.

There are Viewlets on this dashboard that further classify users by application version, OS version, and

phone carrier.



1.2.3 Real User Monitoring

Figure 3. Real User Monitoring

The screenshot above shows a real-user monitoring scenario focusing on users in North America. The

popup on the geographic map is showing a full breakdown of the components and elapsed time for the

user transaction.

Complete tracking of the end user’s experience is provided in real-time. Browsers are automatically

injected with instrumentation without a need to decorate your applications. Insight can find the

bottlenecks that cause a user to have a negative experience and correlate their problems with issues in the

browser itself, applications that the user’s session is dependent on such as a JVMs or databases. Insight

tracks transaction end-to-end starting at the user with a web browser and interacting with application

servers, middleware, databases, and mainframes either local or in the Cloud.



1.2.4 Managed File Transfer (MFT)

Figure 4. Managed File Transfers

The Insight dashboard above has been set up to analyze managed file transfers (MFTs). There are various

Viewlets to track MFTs by application, agent, resource, destination, and status.

Insight tracks all data movement across complex topologies. All MFT transfers are connected with

downstream events from sources including other MFTs, middleware, brokers, and other business

applications. Metrics on MFTs are captured in real-time and evaluated in terms of SLAs and business

objectives. Appropriate notifications are sent out for missed objectives. A search capability is provided

to review past transfer and their attributes. Insight provides a dynamic topology of all MFT transactions.



1.2.5 Application Performance Monitoring

Figure 5. Application Performance Monitoring

The Insight dashboard example above for application performance monitoring (APM) is illustrating how

to monitor the DevOps Jenkins based continuous build-deploy process. The upper Viewlet is an

automatically discovered topology map showing applications and their relationships to other applications

such as “Maven” to “Deploy” as well as resources such an Oracle database and a log4j jar file. It shows

the flow of a deployment process and any exceptions incurred. The lower Viewlet is called the Console

and it opens up when a user drills down into an object on a Viewlet in order to get additional details.

Insight provides deep-dive monitoring of the performance and availability of applications end-to-end

across Web Services, application servers (Java, .Net), middleware, mainframes, and more. Its automation

eliminates the need for constant “eyes-on-screen” monitoring to eradicate false alarms and provide

automated notification of real situations that require attention.



1.2.6 Mobile Analytics

Figure 6. Mobile Analytics

The sample mobile analytics dashboard above is highlighting a scenario where performance is compared

to mobile app version, carrier and device. Using our mobile APIs we can track user experience through

every mobile app screen, analyze user experience and determine which app versions, devices and carriers

deliver the best experience.

Insight provides end-to-end visibility into mobile application behavior and performance for both iOS and

Android. Insight provides RESTful APIs for streaming data and real-time tracking. Mobile apps can

stream their data to AutoPilot Insight, submit interactive queries, and subscribe to real-time analytics.

Crashes can be captured and analyzed for forensic purposes. The APIs enable complete analysis of a

user’s interaction with your applications relating the specific click path through an application correlated

with app version, device information, and even business behavior such as purchasing or cart

abandonment.



1.2.7 Kafka Monitoring

Figure 7. Kafka Monitoring

The Viewlet above shows the auto-discovered, publish-subscribe topology of a Kafka network including

senders, readers, and topics. Each edge (the lines between nodes) has statistics showing average elapsed

time and count. This image shows the topology of a Kafka Sender publishing messages with Topics and

several Kafka Readers subscribed to specific Topics.

A single-point-of-truth is provided to track performance, latency, logs, auditing, and content surveillance.

Insight provides complete message flow analytics relating applications to the messages they publish to

Kafka and the applications that subscribe to them.



1.2.8 Middleware Monitoring and Tracking

Figure 8. Middleware Monitoring and Tracking

The two Viewlets above are from the Sample Middleware dashboards in AutoPilot Insight. The upper

Viewlet is a score card showing MQ event exceptions and the impacted resources. The lower Viewlet is a

stacked bar chart showing the number of MQ INFO, CMIT, and GET events for members of the set

(group) FINTECH_RP43.

AutoPilot Insight provides deep-dive monitoring and tracking of middleware messaging across the IBM

MQ family of products, Active MQ, TIBCO, Kafka, and more. Insight provides auto-discovery of all

components and their state as well as end-to-end tracking of middleware transactions (message flows), the

resources they interact with, and the applications that consume these messages.

1.3 Data

Users can either stream their data or upload a file (section 2.2.2) in order to use AutoPilot Insight to

analyze and present their data.

1.4 jKQL

jKool Query Language (jKQL) is an English-like query and stream processing language for analyzing

machine data in flight (Fast Data) and at rest. It defines the syntax of statements used for manipulating

data in the jKool Data Model. It enables the user to search, filter, group, and count data. It is designed to

be used by both the business user and the data scientist. Use jKQL to analyze anomalies, behavior, flows,

relationships, and patterns in time-series data as it relates to your business.



2 Using AutoPilot Insight

Insight puts your data (streaming or uploaded from a file) in a repository and displays it as a collection of

customized Viewlets grouped into one or more dashboards depending on your needs.

2.1 Accessing Insight

1. Open your internet browser.

2. Enter the URL address provided by your System Administrator and press Enter. The AutoPilot

Insight login dialog box is displayed.

Figure 9. Login Dialog Box

FYI: The login dialog box may display your company logo rather than the Nastel logo.

3. Enter your Login Id and Password and click LOGIN. By default, the Landing Page is displayed.

2.2 Landing Page

The Landing Page is used as an initial screen for AutoPilot Insight novices providing guidance on what

the solution provides as well as an easy-to-use wizard for importing data. Experienced users can skip this

screen and go directly to their dashboards. The Landing Page provides three options:

Explore a Demo (section 2.2.1)

Analyze Your Data (section 2.2.2)

Go to Dashboard (section 2.2.3).

Figure 10. Landing Page

Select to turn off Landing Page.



FYI: The Landing Page is turned on by default. To turn it off, go to the Dashboard screen (Figure 45) and click the Settings icon to display the Settings dialog box. Click Repository, select Landing page Off, and click Save Changes. You can also turn off the Landing Page by selecting Never Show Again on the Landing Page.

Figure 11. Settings Page

2.2.1 Explore a Demo

This option provides walk-throughs of four different business issues where Insight can be used to solve a

problem.

RUM (Real User Monitoring) – illustrates how to determine the root cause of poor end-user

experience. (Go to https://www.youtube.com/watch?v=OuYvkRix6iM to watch a brief use-case

demonstration.)

Order Tracking – illustrates how to trace the flow of an order from order placement through

verification, payment, shipping, and more

IoT – illustrates highlights the Internet of Things (IoT) as used in athletics, specifically basketball

DevOps – illustrates how to analyze the Build and Deploy processes.

Each walk-through starts with an explanation of the problem, the solution, and the steps taken to solve the

problem. To view a demo, select it and click Start a demo.

At the end of each demo, there is an option to load your own data into the example. Click Load your

data and select your file. (Refer to section 2.2.2.)

The following is a walk-through of the Order Tracking demo.

https://www.youtube.com/watch?v=OuYvkRix6iM



2.2.1.1 Order Tracking Demo

Select Order Tracking and then click Start a demo.

Figure 12. Start a Demo



A Viewlet is displayed which shows a topology map of the business milestones. The jKQL query that

produced this Viewlet is shown at the top of the screen.

Figure 13. Order Tracking Demo – Page 1

The healthbar under each icon is color coded to reflect status (green = good, yellow = warning,

red = critical).

To drill into the details of an event, click the icon. Click Next to view the details of the circled

milestone, Order Placed.

Clicking the health bar for Order Placed, produces a pop-up menu (Figure 14) for drill-down

into SLAs and performance metrics for transactions and activities.

jKQL query




Click SLA to see the objective of Elapsed Time <= 2 seconds.

Choose Not Met to see the transactions that did not meet the required performance objective.

Click Next to proceed.



This screen shows the open console where the slow transactions are listed. In this example, a transaction

was selected and topology chosen. Click Next to view the topology.




This screen shows the topology. By clicking the various icons, you can drill down into each event to see

the root cause of problem.


This is the end of this demo. You can either:

Return to the Landing Page by clicking Finish

OR

Upload your own data file into the example by clicking Load your data and following the

prompts.



2.2.2 Analyze Your Data

The following file formats are currently supported:

.csv

.xls

.xlsx

Apache log.

Do the following to upload your file.

1. Click Choose File to select a file to upload.

Figure 17. Upload File

2. In case you selected the wrong file, you have the opportunity to change it by clicking Change and

selecting another file. Then click Next.

Figure 18. Change Your File Selection



3. Review the preview. If your file does not look right, click Advanced to change processing

options.

Figure 19. Preview Uploaded File

Figure 20. Advanced Options for Uploaded File

4. Advanced options allow you to:

Sheet – Select a sheet from your Excel spreadsheet.

First row as header – Specify if the first row is a header row.

Cell range – Specify the columns to upload.

Date format – Specify date format.

After you have made your selections, click Next.



5. This screen allows you to:

Change the name of one or more column headers. You can select from the drop-down list or

type over the existing header name.

Map data imported into Insight to an existing field within the Insight data model.

Alternatively, by selecting the option “new property”, this can be used to import custom data

and label it with a name that has relevance to the user.

Remove a column.

Click Next to continue.

Figure 21. More Advanced Options for Uploaded File

6. This optional screen is used to enrich the data being imported with additional information

including the following:

Application name

Server name

Network address

Data center

Geo address – Click Use current location to populate the field with the latitude and

longitude of your current location.

Figure 22. More Advanced Options for Uploaded File

Click Start import to upload your file.



7. After the upload is complete, the summary screen is displayed. Click Next to continue.

Figure 23. Uploaded File Summary

8. A default set of Viewlets is provided. By default, all Viewlets are selected. (Selected Viewlets

are grayed out.) Click a Viewlet to deselect the ones you do not want. (Deselected Viewlets

appear in full color.) Click Next to add your Viewlets to a dashboard.

Figure 24. Viewlets for Uploaded File

Deselected Viewlet

Selected Viewlet



9. Add your Viewlets to and existing dashboard or create a new dashboard.

ADD TO DASHBOARD – Add your Viewlets to an existing dashboard by selecting a

dashboard from the drop-down list.

CREATE A NEW DASHBOARD – Enter a name for your dashboard and select one, two,

or three columns.

Figure 25. Add Uploaded File Viewlets to Dashboard

Click Finish to display your data in your dashboard.

2.2.2.1 Finding Prior Imports

The AutoPilot Insight Main Menu (Figure 46) has an item called Imports that can be selected to view

which data files have been imported and to optionally generate a new dashboard and Viewlets from the

imported data.

1. After selecting Imports from the Main Menu, a list of imported files is displayed. Select a file

and click Open to start the wizard that will massage the data, import it into a dashboard, and

create new Viewlets.

Figure 26. List of Imports

Select file

Click Open



2. New Viewlets are automatically created. By default, all Viewlets are selected. Deselect any you

do not want to add to your dashboard. Click Next.

Figure 27. Default Viewlets are Created

3. Select an existing dashboard or create a new one by giving your dashboard a name and selecting

the number of columns in the layout. Click Finish to publish your Viewlets in the new or

existing dashboard.

Figure 28. Publish Viewlets to a Dashboard

2.2.3 Go to Dashboard

This option takes you to your dashboard if you have previously created one or allows you to create a new

one (Figure 44).

Click Finish

Click Next

Or create new dashboard

Select existing dashboard



2.3 Sample Dashboards

Sample dashboards are provided. You should review them before creating your own. The Sample Order

Tracking dashboard is shown in Figure 29. The individual Viewlets are described in section 2.3.1

through section 2.3.13.

Figure 29. Sample Dashboard

A dashboard is a collection of Viewlets. You can create more than one dashboard from your data

repository depending on your specific needs.

Right-click the tab to display a menu that allows you to Save or Change Layout (number of columns).

The tab for the default dashboard is marked with a green bar. In this example, Save As and Set As

Default are not available because this is a “sample” repository which is read-only. If it was your

repository, all functions would be available.

Figure 30. More than One Dashboard



2.3.1 Steps in the Order Process Business Milestone

Query: jKQL> get relatives show as topology

Figure 31. Sample Viewlet – Steps in the Order Process Business Milestone

The Viewlet above shows the auto-discovered topology of an Order Process, displayed at the business

milestone tier. Topologies can be shown at the geographical, datacenter, server, application, or milestone

tiers. Each of the blue “chevron”-like icons above represents a specific business milestone. A business

milestone is there to represent the completion of a business objective in the “real-world”. It is defined

based on established criteria, while its completion determines its status. Milestones often form a

sequence or flow. This happens automatically as the analytics engine determines an observed relationship

between them. The colored bars underneath each icon are called a healthbar. It can be clicked on to see

the status of the milestone. The arrows between icons shows data flow between milestones. This is

automatically discovered. The numbers surrounding the arrow show statistics for the relationship

including elapsed time and count.

A topology is often used to see the “flow” of what happened, and when it happened. This is very helpful

in understanding the status of your applications and objectives.



2.3.2 Activity Scorecard Latest Week

Query: jKQL>Get the number of Activities for the last hour where the severity > ‘INFO’ group by ActivityName, location, elaspedtime, severity order by ActivityName, severity desc show as scorecard

Figure 32. Sample Viewlet – Activity Scorecard Latest Week

The Viewlet above is a Scorecard. It is being used in this example to display details about activities that

have an important severity (ones that need attention). The scorecard layout groups activity names in the

first column and their details in the subsequent columns. Each row shows an additional instance of

activities with the same name. Activity names are not unique. They are differentiated by their activity

ID.

Scorecards are most often used as a grouping mechanism to see at a glance the status of a specific

application or activity.



2.3.3 Geomap Events by Location

Query: jKQL> Get relatives show as geomap

Figure 33. Sample Viewlet – Geomap Events by Location

The above Viewlet displays the set of items within a geographic location. Each icon (push pin) represents

a location, e.g., United States and the collection of all of the entities such as applications, activities,

events, and servers in that location. Each of the arrows shows a relationship between entities in one

location with another. The dotted line shows a parent-child relationship (called enclosed) between the

locations, while a solid line would represent an observation of an event in one location sending a message

to an event in another (called send-to).

Geomaps are used when location is important and you want to first start with that and then drill down to

specific applications when troubleshooting a problem.



2.3.4 Orders for the Last Hour that Missed their SLA

Query: jKQL> Get the number of activities for the latest 3 days that did not meet ‘SLA’ group by location, activityname, severity, starttime bucketed by minute show as stackchart

Figure 34. Sample Viewlet – Orders for the Last Hour that Missed their SLA

The Viewlet above is searching for missed SLAs (service level agreements). It is presenting them in a

stacked barchart grouped by name, location, severity, and time.

Stacked barcharts are a powerful way to display a lot of data about the status of something in a very

concise way.

2.3.5 Credit Validation Exceptions

Query: jKQL> Get the Activities from ‘Verify Credit’ that did not meet the 'SLA' group by ActivityName, location, elapsedtime order by ActivityName show as scorecard

Figure 35. Sample Viewlet – Credit Validation Exceptions



The Viewlet above is showing an example of you might track exceptions or errors for specific activities.

Here we are checking for ones that missed their service level agreement (SLA) requirements. A user

would utilize this to find the errors and then drill down into the specifics in the console to try and learn

why. This is part of the forensics process.

2.3.6 Events for Latest Hour by Location

Query: jKQL> Get the number of Events for the latest 4 days where severity > 'INFO' group by location, eventname show as linechart

Figure 36. Sample Viewlet – Events for Latest Hour by Location

The Viewlet above is a line chart showing the trend in important event occurrences. Clicking on any of

the “dots” or points will take the user to the console where they can see additional details about each

event. From there they can compare events or display the topology of an individual transaction.



2.3.7 Application Performance Index Analytics

Query: jKQL> Get activities fields Apdex(ElaspsedTime, 3sec,4.5sec) group by ActivityName, location order by ActivityName show as scorecard

Figure 37. Sample Viewlet – Application Performance Index Analytics

The above scorecard Viewlet is using the statistical function Apdex. AutoPilot Insight comes with a large

library of functions built into it including Bollinger bands, EMA, SMA, Floor, Median, Round, Standard

Deviation, and many more. Apdex stands for application performance index. It defines a method for

reporting and comparing the performance of software applications in order to measure user satisfaction.

Here it is used to determine the experience of users in each geographic area for each activity and its

related applications. A “0” means no users are satisfied, while a “1” means all users are satisfied. A

number in-between shows a mix of satisfaction levels. This is measured in relationship to the target

elapsed time, in this case, between 3 to 4.5 seconds.



2.3.8 SLA Violation Scorecard

Query: jKQL> Get the number of Activities for the latest hour that did not meet the ‘SLA’ group ActivityName, location, elapsedtime order by ActivityName show as scorecard

Figure 38. Sample Viewlet – SLA Violation Scorecard

This Viewlet is a scorecard displaying SLA violations for each activity grouped by location.

2.3.9 Serious Event Distribution

Query: jKQL> Get the number of events for the latest hour where severity > ‘WARNING’ group by location, severity order by severity show as piechart

Figure 39. Sample Viewlet – Serious Event Distribution

The Viewlet above is a pie chart which is often used when counting something and you want to show the

distribution of results for each member of a group or specifically severity in this case. This approach

makes it easy to see where the biggest groups are that may need attention and further forensic analysis.



2.3.10 Elapsed Time for Order Events

Query: jKQL> Get the number of events fields Min(ElaspsedTime), Max(ElaspedTime), AVG(ElaspedTime) group by location show as colchart

Figure 40. Sample Viewlet – Elapsed Time for Order Events

This Viewlet uses the functions: min, max, and average as applied to elapsed time for events.



2.3.11 Exponential Moving Average for ElapsedTIme

Query: jKQL> get events compute EMA(ElaspsedTime, 20) show as linechart

Figure 41. Sample Viewlet – Exponential Moving Average for ElapsedTIme

Here we are computing an exponential moving average (EMA) to chart elapsed time over a window of

time. EMAs are used with trends and enables one to see the rate of change between one data point and

the next.



2.3.12 Anomaly Monitor

Query: jKQL> Get the number of events group by StartTime bucketed by days show as anomalychart

Figure 42. Sample Viewlet – Anomaly Monitor

This Viewlet, which is called an anomaly chart, is using the function Bollinger Bands to automatically

detect anomalies in the number of events per day.

2.3.13 Function Analysis

Query: jKQL> Get Activities fields StdDevPop(properties(‘OrderAmount’)), StdDevSample(properties(‘OrderAmount’)), VariancePop(properties(‘OrderAmount’)), VarianceSample(properties(‘OrderAmount’)) for this year group by props(‘COUNTRY_NAME’) show as scorecard

Figure 43. Sample Viewlet – Function Analysis



This Viewlet is an example of using standard deviation on the order amount field. Standard deviations

are used to determine how far a value is from the expected value or mean and can illustrate the volatility

of this value over time.

2.4 Create a Dashboard

A dashboard is a collection of Viewlets. Your data repository can have more than one dashboard. Each

dashboard can be displayed by clicking its tab on the top of the screen. Your default dashboard is

indicated by a green bar on the top of the tab. (See Figure 45.)

Users create multiple dashboards as a way of grouping different data or analytics. While all could be on a

single dashboard within the same repository, it can be more convenient to break them up by separate

dashboard tabs.

After clicking Go to Dashboard from the Landing Page (Figure 10), the Create new Dashboard dialog

box opens.

FYI: You can also create a new dashboard by opening the Main Menu (Figure 46) and selecting Dashboard > Create.

Figure 44. Create New Dashboard Dialog Box



To create your dashboard:

1. Enter a name for your dashboard.

2. Select the number of columns.

3. To create a set of default Viewlets, select Generate initial viewlets.

4. Click Create. Your dashboard has been added. Figure 45 shows a new dashboard with a set of

default Viewlets displayed as thumbnails. The queries are Get activity, Get Count, and Get

Snapshot. By clicking the Viewlet thumbnail, the Viewlet, in the default table format, opens in

the console at the bottom on the screen.

Figure 45. Default Viewlets

The upper portion of the Viewlet above is a Summary Viewlet. It is used when counting the number of

some object like an event, activity, or application and presenting the count in a Summarized view. The

place on the screen where Summary Viewlets reside is called the Summary Dock. It can be closed and

default to closed when no Summaries are defined for that dashboard.

Default Viewlets

Activity Count Viewlet Details

Main Menu

(Figure 46)

Viewlet Menu (Figure 51)

Default dashboard indicated by green bar.



2.4.1 Main Menu

The Main Menu (Figure 46) provides the functionality described in Table 1.

Figure 46. Main Menu

Table 1. Main Menu Functions

Viewlet – Expand to access the following:

Create Allows you to create a new Viewlet.

Open Opens the Open Viewlet dialog box (Figure 54) where you can select one or more Viewlets to open.

Import/Export Opens the Import/Export Viewlets dialog box (Figure 56 and Figure 57) where you can import or export a dashboard. Must be .json file.

Dashboard – Expand to access the following:

Create Opens the Create new Dashboard dialog box (Figure 44) where you can create a new dashboard.

Open Allows you choose an existing dashboard and open it. If there are not any additional dashboards, the menu collapses.

Save Allows you to save your dashboard.

Save As Allows you to save a copy of your dashboard.

Change layout Allows you to change the layout to one, two, or three columns.

Import/Export Opens the Import/Export Dashboards dialog box where you can import or export a dashboard. Must be .json file.

Imports Opens the List of Imports dialog box which lists all the data files that have been imported. Select a file to generate a dashboard and Viewlets by following the steps in the wizard (section 2.2.2.1).

Landing Page Takes you to the Landing Page (Figure 10).

Profile Allows you to change your password.

About Displays various information about the application.



2.4.2 Additional Functions

The following functions are available from the dashboard by clicking the appropriate button.

Questions – Allows you to send your questions via email to the Insight

Support Team. Clicking this button opens a blank email message addressed to

[email protected].

Collectors – Takes you to a web page that provides a list of open source

collectors.

Leave Feedback – Takes you to an online survey about the product.

The next step is to add Viewlets (Figure 47). Each dashboard can have multiple Viewlets.

2.5 Create Viewlets

Viewlet types include: tables, graphs, heat maps, score cards, consoles, and topology. Templates for all

Viewlet types along with jKQL queries are provided on the Sample Dashboard.

From the dashboard, click the Viewlet button (Figure 45) or click the Menu icon, select Viewlet to

expand the menu, and select Create to open the Create Viewlet dialog box (Figure 47).

Figure 47. Create Viewlets Dialog Box

Select Create Viewlet with jKQL and click Create.

2.5.1 Enter a jKQL Query

1. In this example, the query Get Event is entered. As you type, suggestions are provided in a drop-

down list.

Figure 48. Enter a jKQL Query

Suggestions are provided as you enter your query.



Figure 49. Name Your Query

3. Enter a name for you query. In this example, My First Query was entered.

4. Click Create. Your first Viewlet is added to your dashboard. Notice the Tear Off button .

Click it to open the Viewlet in a new window.

The tear off feature is helpful in the datacenter where you may wish to display an AutoPilot

Insight screen on a large monitor so all can see the current status. One possible usage in that case

would be to show a large screen of Summary Viewlets giving a high-level view of the status of

your environment.

An alternative use case might be for an administrator or developer with multiple screens on their

desktop. They might have the full dashboard on one screen and a tear off of a specific Viewlet

they are configuring on the other.

Figure 50. First Viewlet



2.5.2 Editing a Viewlet

Click the Viewlet Menu icon and select Edit Viewlet.

Figure 51. Edit Viewlet Menu

Options are different for each display type. Make changes and click Update.

Figure 52. Edit View Dialog Box

Click Viewlet Menu icon.



2.5.3 Edit Query

The query line becomes an editable field after you click the edit query icon or you can simply click the

query line. Make your changes and click the Refresh Viewlet icon. The QB (jKQL Query On/Off)

button is on by default. When it is on, as you edit, you will be prompted with suggestions as in

Figure 48, Enter a jKQL Query. When it is off, you will be prompted with up to 10 prior queries that

match your input.

Figure 53. Edit Query

2.5.4 Other Functions on the Edit Viewlet Menu

Other Edit Viewlet Menu (Figure 51) functions are described below.

Save Viewlet – Allows you to save any changes to the Viewlet.

Save as Viewlet – Allows you to create and save a copy of the Viewlet with a new name. The

new Viewlet can be found on the Open Viewlet dialog box (Figure 54) and added to any

dashboard.

Remove Viewlet – Allows you to remove the Viewlet from the dashboard. A dialog box opens

asking you to confirm the removal. The Viewlet is not deleted and can be restored by opening

the Main Menu, select Viewlet, select Open (Open Viewlet dialog box opens), select the

Viewlet, to be restored, and click Open.

Figure 54. Open Viewlet Dialog Box

Delete Vielwlet – Allows you to delete the Viewlet. A dialog box opens asking you to confirm

the deletion.

Export to CSV – Allows you to download the data to a .csv file.

Click edit query icon and edit query.



2.5.5 Import/Export Viewlets

This function allows you to import/export a Viewlet in the form of a .json file.

1. From your dashboard, click the Menu icon and expand the Viewlet menu.

Figure 55. Viewlet Menu

2. Select Import/Export.

Figure 56. Import Viewlet Dialog Box

3. To import a Viewlet:

a. Select the Import tab (Figure 56).

b. Click Choose File and select your .json file.

c. Click Import. The Viewlet is added to the Open Viewlet dialog box (Figure 54) and can be

added to any dashboard.



4. To export a Viewlet:

a. Select the Export tab (Figure 57).

b. Click Choose File and select the Viewlets you want to export or Select All.

c. Click Export. The Viewlets are downloaded in .json file format.

Figure 57. Export Viewlet Dialog Box



2.6 Settings

The Settings function allows you to:

Customize the date range (Figure 58)

Configure your dashboard (Figure 59)

Set a default repository and turn the Landing Page on or off (Figure 60).

To access settings, go to the Dashboard screen (Figure 45) and click the Settings icon to display the

Settings dialog box. The various functions are displayed on the menu bar on the left of the screen.

2.6.1 Date & Time

There are two ways to set your time period:

Show – Select a time period from the drop-down list

Or Use Customize Date Range – Select Custom Range from the Show drop-down list to

activate the From and To input fields. Click the input field and use the date/time widget to set

your date range.

Figure 58. Settings Dialog Box – Date & Time



2.6.2 Dashboard

This screen allows you to:

Change the layout by selecting one, two, or three columns

Change the name of the dashboard by clicking the pencil icon

Delete the dashboard by clicking the trash can icon and clicking Yes on the confirmation dialog

box.

Turn the Summary Console on/off. If a user has no need to view Summary Viewlets, then they

have the option to hide their display. The default action of the dashboard is to show Summaries if

they exist. Turning off the Summary Console would hide existing Summary Viewlets. If the user

does not have any Summary Viewlets, then the Summary dock would default to the closed

position.

Figure 59. Settings Dialog Box – Dashboard



2.6.3 Repository

This screen allows you to:

Set a Default repository ID by selecting it from the drop-down menu

Turn the Landing page on/off.

Figure 60. Settings Dialog Box – Repository

3 Functions

There is a library of functions that are available for use in AutoPilot Insight. They include the following:

Abs DayOfWeek Power

AllPathsBetween EMA RelatedAnomalies

AllPathsToFrom Exp Replace

Anomaly Extrapolate Right

AnomalyDeepDive FindIn RogueEdges

AnomalyDeepDiveRogueEdges Floor RootCauseAnalysis

Apdex HoltWintersPrediction Round

Avg Left SMA

Avg Length ShortestPath

BBands List Sqrt

Cast Ln StdDev

Ceil Log10 StdDevSample

Close Lower StrAt

Coalesce Max SubStr

Concat MaximumFlow SubStrRE

ConcatWS Median Sum

Count Min Tokenize

DateAdd Now UUID

DateAdjust Open Upper

DateDiff Position ValueAt

DateExtract PositionRE Var

VarSample



4 Using jKQL

4.1 Get

The Get statement is used for retrieving items from the database.

Examples:

To get default fields for all Activity items:

Get Activities

To get all fields for all Activity items in Set “Purchasing”:

Get Activity Fields All from 'Purchasing'

To get the number of Activity items in Set “Purchasing”:

Get number of Activities from 'Purchasing'

To get the number of Activity items in Set “Purchasing” that started today:

Get number of Activities from 'Purchasing' for today

To get the 10 longest running Activities in Set “Purchasing” that started today:

Get top 10 Activities from 'Purchasing' for today sort by ElapsedTime

desc

To get the number of Activities in "Payment" last week grouped by their start time:

Get number of Activities from Payment for last week group by starttime

To get the number of Activities in Set “Purchasing” for each Activity status for the last week:

Get number of Activities from 'Purchasing' for last week group by

Status



To get the number of Activities in Set “Purchasing” that met all objectives:

Get number of Activities from 'Purchasing' that met all objectives

To get the number of Activities in Set “Purchasing” that did not meet some objectives:

Get number of Activities from 'Purchasing' that have not met all

objectives

To get the number of Activities in Set “Purchasing” that did not meet objectives “A” and “B”:

Get number of Activities from 'Purchasing' that have not met objectives

'A','B'

To get Activities in Set “Purchasing” that did not met objectives “A” and “B” from set “Web Purchases”:

Get Activities from 'Purchasing' that have not met objectives 'A','B'

from 'Web Purchases'

4.2 Show as

At the end of the query, you can specify the format you want the results displayed in by using "show as".

Show as Table is the default. Other options are piechart, barchart, colchart, stackchart, topology,

geomap, and histogram.

Get relatives show as topology

Enrich your queries with additional items such as:

Time ranges – month, day, hour Get events for this month

Group by – creates a row for each unique set of values for columns being grouped on. Get events fields location where eventname contains ‘rder’ group by

location show as barchart

Buckets – Bucketing allows multiple “group by” result rows to be combined into a single result

row. Used when a "group by" statement returns too much data. Bucketing can only apply be

applied to INTEGER, DECIMAL, TIMESTAMP, and TIMEINTERVAL data types.

Locations – Geolocation Get Event for This Month where Location =’London, England’

Sort by – Sorting criteria Get Activities from 'Purchasing' for today sort by ElapsedTime desc

Latest – This differs from "last" which could return nothing as there may not be any events in the

last 5 days

Get number of events for latest 5 days group by starttime bucketed by

day, severity show as stackchart

autopilot insight user’s guide - nastel · autopilot insight user’s guide confidentiality...

Documents