Hands-On LabExploratory Testing using Microsoft Test Manager 2015Lab version: 14.0.25123.0

Last updated: 5/17/2016

TABLE OF CONTENT

EXPLORATORY TESTING USING MICROSOFT TEST MANAGER 2015.......................................................................1

OVERVIEW..................................................................................................................................................................3Prerequisites.......................................................................................................................................................3About the Fabrikam Fiber Scenario.....................................................................................................................3Exercises..............................................................................................................................................................3

EXERCISE 1: EXPLORATORY TESTING INTRODUCTION...........................................................................................................4Task 1: Starting Exploratory Testing...................................................................................................................................4Task 2: Working with the Test Runner Client.....................................................................................................................8Task 3: Creating Bugs and Test Cases...............................................................................................................................17Task 4: Continuing the Exploratory Testing Workflow.....................................................................................................20Task 5: Including Repro Steps..........................................................................................................................................22Task 6: Viewing Results of Exploratory Testing................................................................................................................25

EXERCISE 2: EXPLORATORY TESTING WITH THE CHROME EXTENSION....................................................................................26Task 1: Installing the Chrome Extension..........................................................................................................................26Task 2: Exploratory Testing with the Chrome Extension..................................................................................................28Task 3: Creating Bugs with the Chrome Extension...........................................................................................................32

OverviewIn this lab, you will learn about the exploratory testing capabilities provided in Microsoft Test Manager 2015. You will learn about how exploratory testing (also called XT, or agile testing) has become a first-class experience in the toolset. This allows a tester to more flexibly test the underlying software without relying solely on formal test cases. Meanwhile, Microsoft Test Manager will continue to capture rich diagnostics about the application being tested which can be delivered to the development team if a bug is discovered. You’ll also learn about some of the other fit-and-finish features in Microsoft Test Manager, such as the ability to use rich text when authoring test cases.

Prerequisites

In order to complete this lab you will need the Visual Studio 2015 virtual machine provided by Microsoft. For more information on acquiring and using this virtual machine, please see this blog post.

About the Fabrikam Fiber Scenario

This set of hands-on-labs uses a fictional company, Fabrikam Fiber, as a backdrop to the scenarios you are learning about. Fabrikam Fiber provides cable television and related services to the United States. They are growing rapidly and have embraced Windows Azure to scale their customer-facing web site directly to end-users to allow them to self-service tickets and track technicians. They also use an on-premises ASP.NET MVC application for their customer service representatives to administer customer orders.

In this set of hands-on labs, you will take part in a number of scenarios that involve the development and testing team at Fabrikam Fiber. The team, which consists of 8-10 people, has decided to use Visual Studio application lifecycle management tools to manage their source code, run their builds, test their web sites, and plan and track the project.

Exercises

This hands-on lab includes the following exercises:

1. Exploratory Testing Introduction

Estimated time to complete this lab: 30 minutes.

Exercise 1: Exploratory Testing IntroductionIn this exercise, you will learn about some of the exploratory testing capabilities of Microsoft Test Manager 2015 that make exploratory testing easier on the software tester.

Task 1: Starting Exploratory Testing

1. Log in as Adam (VSALM\Adam). All user passwords are P2ssw0rd.

2. Launch Microsoft Test Manager from the taskbar. Microsoft Test Manager allows testers to work with test plans, author and organize manual test cases, execute test cases, file bugs, and post results back to Team Foundation Server.

3. Connect to the FabrikamFiber team project. You will need to press the Home button and the Change Project button to get to the team project selection window (as shown in series of screenshots below).

Figure 1Home button

Figure 2Change Project button

Figure 3Connecting to the FabrikamFiber team project

4. In the Testing Center window, click Add to create a new test plan.

Figure 4Adding a new test plan

5. Enter “Exploratory Testing” as the name of the new plan and then click Add.

Figure 5Adding a new test plan

6. Click Select plan to open it.

Figure 6Opening the new exploratory testing plan

7. You should now be in the Testing Center and connected to the Exploratory Testing test plan.

Figure 7Testing Center showing exploratory testing plan

8. Right-click the Exploratory Testing test plan node and select the Explore option from the context menu.

Figure 8Starting exploratory testing

Note: You may notice a message stating that desktop screen recording is currently disabled. This is simply an informational message that informs the tester that the test settings currently in use have not included the Screen and Voice Recorder diagnostic adapter. If desired, the test settings can be modified in Plan | Run Settings underneath the Manual runs section.

Task 2: Working with the Test Runner Client

1. Click the Settings button in the bottom-right corner of test runner window.

Figure 9Location of test runner settings button

2. Note that we can modify screenshot settings and select an audio recording device here. Since this virtual machine does not support audio, we will not attempt to make a selection. Press Esc to continue.

Figure 10Viewing current settings

3. Click Start to start an exploratory testing session.

Figure 11Starting the test runner for exploratory testing

4. Launch Internet Explorer and load the Fabrikam Fiber intranet portal by clicking the FF Intranet link in the favorites bar. You should now see the dashboard page.

Figure 12Launching the Fabrikam Fiber intranet portal

5. Select the Tickets tab on the Fabrikam Fiber intranet portal site to view all the service tickets.

Figure 13Navigating to Tickets screen

6. Click one of the service ticket links to view the details.

Figure 14Selecting a service ticket to view details

7. Scroll to the bottom and note that the Created By and Assigned To fields currently show appropriate values.

Figure 15Viewing service ticket details

8. Click Edit near the bottom of the ticket details page (you may need to scroll down).

Figure 16Editing a service ticket

9. Note that the Created By property still shows an appropriate value.

Figure 17Edit page for service ticket

10. Click Save. As expected, the Created By and Assigned To properties are still the same as they were before.

Figure 18Saving ticket details

11. Click Delete to delete the ticket.

Figure 19Deleting a service ticket

12. In the Delete confirmation page, note that the Created By and Assigned To properties are now set to None, which is not what we were expecting. In the comments box of the test runner, enter the text “CreatedBy and AssignedTo are blank when viewing Delete confirmation page.” Press the Enter key twice to leave room for a screenshot.

Figure 20Delete confirmation page showing unexpected values

13. Click Add Screenshot to add a screenshot of the Delete confirmation page to provide some context.

Figure 21Location of screenshot button

14. By default, this will allow you to draw a rectangle to define the area that you wish to capture. Hold the left mouse button down and move the mouse until you highlight the important part of the Delete confirmation page.

Figure 22Selecting the area of the screenshot

Figure 23Test runner window showing inserted screenshot

Note: You can double-click the screen capture to edit it in a paint program to provide additional annotation if desired. By default, MS Paint is used, but you can use your tool of choice by configuring this in the settings dialog that you saw at the beginning of this exercise.

Task 3: Creating Bugs and Test Cases

1. Click Create Bug to file a new bug.

Figure 24Creating a new bug

2. For the title, enter “CreatedBy and AssignedTo are blank when viewing Delete confirmation page” and assign the bug to Julia Ilyiana. Note that your observations, repro steps and diagnostic data are automatically included.

3. Click Save and create test to finish filing the new bug.

Figure 25Creating a new bug as the result of exploratory testing

4. In the New Test Case window, enter a title of “Confirm that AssignedTo and CreatedBy fields appear on Delete confirmation page.”

5. In the Steps section of the New Test Case window, locate the row that starts with “Click ‘Delete’ link” and add the following text in the Expected Result column:

AssignedTo and CreatedBy fields should have appropriate values when viewing Delete confirmation page

Figure 26Adding notes to the Expected Result column for a step

6. Click Save and close.

Figure 27Save and close the new test case

Task 4: Continuing the Exploratory Testing Workflow

1. After the new test case is saved, we are automatically returned to our exploratory testing task and can continue looking for more bugs. Let’s start by clearing out the observations from the previous bug that we just saved. Press Ctrl+A and then the Delete key.

2. In the Internet Explorer window, select the Dashboard tab.

Figure 28Navigating to the dashboard

3. Click one of the service ticket links to view the details.

Figure 29Selecting a service ticket to view details

4. Click Escalate (you may need to scroll to the right).

Figure 30Escalating a service ticket

5. The result of the attempt to escalate the service ticket is a HTTP 404 server error.

Figure 31Escalating the service ticket results in an error

6. Click Create Bug to file a new bug.

Figure 32Creating a new bug

7. For the title of the bug, enter “Escalate service ticket results in a HTTP 404” and assign it to Julia Ilyiana.

Figure 33Creating a new bug as the result of exploratory testing

Task 5: Including Repro Steps

1. In the Steps To Reproduce section of the new bug, note that there is a link that allows you to easily scope the repro steps. Click Change steps.

Figure 34Changing the scope of the repro steps

2. In the Change Steps window, note that the steps that you took most recently are already highlighted (those starting with the click on the Dashboard hyperlink). This represents the recent actions taken to reproduce the error we just saw. You could optionally change the selected steps if needed.

3. Click Select to continue.

Figure 35Selecting the repro steps

4. Click Save and create test.

Figure 36Creating a new bug as the result of exploratory testing

5. In the New Test Case window, enter a title of “Ensure that ticket escalation page works” and then click Save and close.

Figure 37

Save and close the new test case

Task 6: Viewing Results of Exploratory Testing

1. Click End testing in the test runner window.

Figure 38End the exploratory testing session

2. In the session summary window for the exploratory test, click Save and Close to return to the Plan tab of the Testing Center.

Figure 39Save and close the exploratory testing session

3. Note that there are now two formal test cases that are part of the Exploratory Testing suite. By formally capturing these test cases, we can ensure that we test for them in the future. This enables us to use a blend of approaches, both exploratory testing and formal testing, to improve our overall application quality.

Figure 40Formal test cases were created as part of the exploratory testing session

Exercise 2: Exploratory Testing with the Chrome ExtensionTask 1: Installing the Chrome Extension

1. Install Google Chrome from http://google.com/chrome.

2. Launch Chrome from the taskbar.

3. Open the Visual Studio Marketplace page (https://marketplace.visualstudio.com). Select the Visual Studio Team Services option.

4. Search for “Exploratory testing” and select Exploratory Testing extension from the search results to open its details page.

5. Click on Install button on the details page. This will take you to the chrome web store page of the extension. Click on Add to Chrome option.

6. Confirm the installation when asked.

7. To open the extension, click the extension icon that will appear on the right of the address bar. The welcome notification informs you that the last 15 actions that you take on the Chrome browser will be captured by the extension and automatically added as image action log in the work items created using the extension. Click Ok, Got it! to confirm and move to the Connection Settings page.

8. The extension can be used in two modes: Connected and Standalone mode. If you have Team Foundation Server (TFS) connection or a Visual Studio Team Services (VSTS) account, select Connected mode. Standalone mode is for users who don’t have TFS (or VSTS account) and want to use the extension to file bugs and share the report with their team.

9. Since you have a TFS instance to work with, select the connected mode and enter the TFS server name “vsalm” in the Server URL text box.

10. After connecting to TFS, you will need to select the team you want these efforts associated with. Select the FabrikamFiberCollection and type “web” to search for the web team. Select the Fabrikam Fiber Web Team when available. Now all of your work item creation and searching will be scoped to the area path this team belongs to (“Development”). Click Save to continue.

Task 2: Exploratory Testing with the Chrome Extension

1. To start an exploratory testing session, click the Start session button from the extension toolbar.

2. Navigate to http://intranet.fabrikam.com.

3. Click the first dashboard ticket to view it.

4. Click the Escalate button to attempt to escalate the ticket. This will lead to a 404 page, which is a bug.

5. Click the Exploratory Testing icon button and click the Capture screenshot button.

6. Select a section of the screen and change the name of the screenshot to something more descriptive. Note that there are a variety of drawing tools available to really help you get your point across. Click the Blur area button.

7. Highlight a section of the screenshot, which will be blurred out to become illegible. This makes it easy to report bugs and create other work items without risking sensitive information. Click the Save screenshot button to add it to the current session’s timeline.

8. Click the Add note button to start a note. Type in something insightful and click Save to save the note to the session’s timeline.

9. You can also record screen video to capture flaky issues like flickering, abnormal behavior of the web application etc. that are difficult to capture using screenshots alone. To record your screen click the Record screen button and click Start recording.

Select the appropriate screen you want to record, in this case the 404 page and click Share. The recording for the screen selected has started.

10. Slowly highlight part of the page. This task doesn’t need to be critically accurate, but is rather just something to produce video to review later on.

Click the Stop recording button on the extension to stop recording the screen. The screen recording gets saved.

11. Click on the View session timeline button. You can see that all the content – screenshots, notes and videos you’ve captured are available in the current session’s timeline. You can open the screenshots, read your notes or play the video from here.

Task 3: Creating Bugs with the Chrome Extension

1. Click the Create bug icon button followed by the expanded Create bug text button.

2. Enter “Escalate ticket results in 404” as the name of the bug. All the screenshots and videos captured earlier are already a part of the bug. In addition to this the image action log (user actions) has also been inserted for you so that it’s easy for others to easily reproduce the issue. Click Save to save the bug to TFS.

3. Another great feature of this extension is its ability to view similar bugs and thus reduce redundant issues. Let’s say you were another tester who happened to find this same bug and weren’t aware that it was already logged. Click the Create bug icon button followed by the expended Create bug text button again to start a new bug form.

4. As you type the title “404 escalate” for the bug you will notice that the extension has detected that there is already another bug in the backlog with a similar title. Click 1 similar to view the similar bug.

5. This bug appears to be the same as the one we were about to file, so select it and click Edit to edit the existing bug rather than to file a new bug.

6. At this time the current repro steps are added to the image action log. In practice, you would now review the existing repro steps to determine if you located this bug using a different method, in which case you would update the contents to indicate that your method is an alternative repro. Click Save to save the bug.

7. Click the View session timeline icon button and click the last created Bug link to open it.

8. Now back on the TFS site, you can update the bug as needed, such as by assigning it to someone or adjusting the severity.

To give feedback please write to VSKitFdbk@Microsoft.scom

Copyright © 2023 by Microsoft Corporation. All rights reserved.