Tutorial: Nuevosoft Test Manager

This tutorial will help you explore main features of Nuevosoft Test Manager. The tutorial is designed such that it can be completed in less than 10 minutes.

1.0 Introduction

Suppose you are building a web based mail application, "NuevoMail" (like Yahoo! mail or Gmail) and would like to test the login feature of NuevoMail on Internet Explorer 6.0 and Firefox 3.0. This tutorial will show you how to use all the main features of Nuevosoft Test Manager that are required for achiveing this goal. It will show you how to:

  1. Define products;
  2. Define tests for the products;
  3. Define testsuites;
  4. Define test platforms;
  5. Define testruns;
  6. Assign testruns to testers and to upload test results in Nuevosoft Test;
  7. Use test analytics i.e. dashboards and reports to gauge effectiveness of our testing.
2.0 Details
2.1 NuevoMail Feature

NuevoMail has login screen that authenticates users. The same screen also has "Forgot password" link which will not be tested during this tutorial.

2.2 Testcases of login feature

This login feature can be tested with the following testcases When the user enters:

  1. Correct login name and correct password, the login should succeed.
  2. Correct login name and wrong password, the login should fail.
  3. Wrong login name and wrong password, the login should fail.
  4. No login name and no password, the login should fail.
2.3 Platforms

Assume our login screen needs to work on Windows XP as XP is widely used and it should also work on Firefox on any platform as Firefox implements similar alogorithm on all platforms.

  1. Internet Explorer 6.0 on Windows XP
  2. Firefox 3.0 on any Operating System. This implies OS is irrelevant for Firefox
3.0 Procedure

This procedure should be followed step-by-step and you are encouraged to use the same product, version names etc. as given here so that data set that is already created can be used to build this tutorial.

In this section we define product under test. This definition of product includes its version, features that are included in that version and testcases that test these features.

3.1 Define product
  1. Click Products in the left bar.
  2. Click Add. A form gets displayed at the bottom of the screen.
  3. Add "NuevoMail" in Name field. Add description. Leave status as New and "Propagate Versions" selected.
  4. Click Save. Notice the product NuevoMail appears in the Product list.
3.2 Define product version
  1. Click "Nuevomail". Click Versions tab.
  2. Click Add.
  3. Enter 1.0 for "Version ID". Leave optional fields unmodified
  4. Click calendar icon next to Planned Release Date. Choose a date
  5. Click Save. Notice that version 1.0 appears in the version list.
3.3 Define features and testcases
  1. Click Features and Testcases tab
  2. Download spreadsheet template in which to add features and testcases by clicking on Export Template. A pop-up comes up and after processing it provides a link at the bottom to download.
  3. Download this spreadsheet and save it on your machine. Fill out the sheets with details. This filled template then can be imported into the system.

The filled template should look like this sample spreadsheet. The sample itself can be uploaded if you have followed instructions up to this point verbatim. Alternately, to simplify filling out details into just downloaded sheet, cut-paste features from sample Features tab and similarly Testcases from Testcases tab into downloaded sheet.

Save this spreadsheet at a known location on your machine and upload them to the system.

3.4 Upload features and testcases
  1. Click Browse. Select the just saved spreadsheet file for features and testcases
  2. Click Upload
  3. Click Save
You have now defined features and testcases of the NuevoMail login screen.
3.5 Define testsuite

Testsuite is a collection of testcases. We now explore how to define a testsuite.

  1. Click TestSuites tab
  2. Select version as "1.0"
  3. Click Add. A form gets displayed at the bottom of the browser
  4. Enter "My functional testsuite" in Name field. Add description
  5. Click Export Testcases and download spreadsheet to your machine. This spreadsheet contains all testcases the system contains for your product. Delete testcases that are not to be included in the testsuite by deleting entire rows from the spreadsheet. Save the spreadsheet. The filled spreadsheet should look something like this sample spreadsheet. (The sample itself can be uploaded if you have followed instructions up to this point verbatim.)
  6. Click Browse and select this file containing testsuite definition
  7. Click Upload
  8. Click Save. Notice testcases count changes to 4 from 0 for "My Functional testsuite"
3.6 Define build

Define build for which we will run tests

  1. Click Builds tab
  2. Select version 1.0
  3. Click Add. Leave all fields as default
  4. Enter build1 for "Build ID"
  5. Enter date using calendar icon. Leave all other fields with default value
  6. Click Save. Notice build1 appears in the list of builds.
3.7 Define platform and platform elements

When we test on Firefox and Internet Explorer we would like track their results separately so that if a feature fails on one browser (platform) but succeeds on another, we can quickly zero in on such failures through reports and dashboards. Hence now we need to define platform first. (on a shared demo system these platform elements and platform definitions may have been created already and hence nothing may need to be done for these steps).

3.7.1 Define platform elements
3.7.1.1 Define platform element category
  1. Click Platforms in the left pane.
  2. Make sure that Platform elements tab is selected.
  3. Click Add platform element. A form gets displayed at the bottom of the screen.
  4. Click Add new category link displayed on top-rigth corner in the form. A window pops up titled "Schema Designer"
  5. Select node "CONFIGURATIONELEMENTTYPE" by clicking on the radio button and Click Add value
  6. In the pop-up that comes up, add "My browser" as category and some description. Click Save. Notice that "my browser" has appeared in the hierarchy. (optional - If you had defined another category, say "my operating system, besides "my browser", then you would follow a similar procedure for defining the operating system category. )
Close the window with the "Schema Designer" title and switch back to the main browser window.
3.7.1.2 Define platform element values
  1. 1. Click Add platform element
  2. Select category "my browser"
  3. Enter "IE6.0" in Name field and add description
  4. Click Save
  5. Click Add platform element
  6. Select "my browser" in the category
  7. Enter "Firefox 3.0" in Name field and add description
  8. Click Save

We now have 2 elements of browser platform category defined. Similarly create Platform Element for "operating system" and name it "Windows XP"

We now need to define platforms.

3.7.2 Define platforms
  1. Click Platforms tab
  2. Click Add platform button. A form gets displayed at the bottom of the screen
  3. Enter "My Firefox" in Name field . Add description
  4. Select "Firefox 3.0" in the pull down for "my browser" in the Platform Elements section. (Optional - If you need to use other attributes then they all should be selected here.)
  5. Click Save
  6. Click Add platform button. A form gets displayed at the bottom of the screen
  7. Enter "My Internet Explorer" in Name field. Add description
  8. Select "IE6.0" in the pull down for "my browser" in the Platform Elements section. (Optional - If you need to use other attributes such as operating system then they all should be selected here.)
  9. Click Save.

You have now defined two platforms: "My Firefox" and "My Internet Explorer"

3.8 Define testruns:

A testrun is a collection of tests assigned to a tester to perform on a specific version of a product using a specific platform for a specific build.

  1. Click Testrun in the left bar
  2. Click Add. A form gets displayed at the bottom
  3. Click Select next to Product label. A pop up comes up
  4. Select product "NuevoMail" after expanding parent product (your name). Click Save to dismiss pop-up
  5. Select Version 1.0 and Build as "build1"
  6. Enter "my testrun for build1" for "Test Run ID"
  7. Enter some descriptive name and description
  8. Select "Status" as assigned. If the status is not set as assigned, then the testrun will not show up in the list of tests to be ready for execution when user clicks "My Tests" link.
  9. Select platform as "My Firefox"
  10. Click Select tester. A pop up comes up
  11. Click All
  12. Click Select button next to the name you want to assign tests to. Click Done. The pop-up is removed
  13. Click check box for "My functional testsuite" in the Testsuites section at the bottom of the screen
  14. Click Save. Notice that a new testrun is added to the Testruns list

At this time all the tests that need to be run by the tester are assigned and when he/she logs in, can see all those tests in "My tests"

3.9 Execute testrun:
  1. Click My tests in the left bar. (Administrator can view all testruns)
  2. Click My testrun for build1 in "Test Run ID" column. A form gets displayed at the bottom of the browser
  3. Click Export all testcases link. Save the spreadsheet that is downloaded
  4. Edit the spreadsheet to indicate what tests failed, passed etc and save it. This file should look something like this sample spreadsheet. (If you followed the instructions verbatim, you can even upload the sample spreadsheet.)
  5. Click Browse. Select the just saved spreadsheet with updated data. Click Upload
  6. Click Save. Progress window indicates the stages of uploading

You have now run all the tests that are needed to explore test analytics.

4.0 Check test analytics

Test analytics gives you complete and clear visibility in this testing activity.

4.1 Dashboards
  1. Click Test Reports in left bar
  2. Click Regenerate all dashboards link at the bottom of the page. Click OK on confirmation box. Close status window
  3. Click Test analytics in left bar
  4. Click Tests tab

Notice the Feature coverage dashboard is showing 2 features as not having testcases. Notice also that test coverage shows proper data broken down by pass, fail and blocked etc.

4.2 Reports

Here we will run just one of the many reports to see the results of our test run.

  1. Click Test reports in the left bar
  2. Click Consolidated test report
  3. Click Select next to product, a pop up comes up. After expanding parent product (your name). Select NuevoMail. Click Save.
  4. Click Generate

Notice that the report accurately shows results of the test run such as passed, failed, blocked and not done test cases.