This chapter gives you an overview of the Bugzilla Test Runner (BTR) capabilities, reviewing briefly its GUI.
The main page of BTR lists all the products defined in your Bugzilla database. Test plans are associated with products.
Clicking on a product name, BTR presents you with a list of the test plans defined for that given product.
Initially, your set of test plans is empty. Click on Add to create a new plan.
You will be asked for a name and the test plan document. Both can be modified later on.
After the test plan is added, go to the edit test plans page. You will see the product has one test plan defined. This page is your starting point when working with test plans.
To edit the test plan, click on the plan's name.
A test plan consists of the following major parts
The plan document is where you may publish descriptions of what your tests are supposed accomplish. Information on this page is customizable, choose what you like or feel is appropriate. A Plan Document typically includes information such as the plan's purpose, scope, software and hardware requirements, etc.
BTR itself does not enforce any defined structure for a plan document. The document is stored and displayed in HTML. You can insert links to external documentation, or to Bugzilla, or other BTR pages.
The core of each test plan is the test cases listing. In BTR, test cases are grouped into functional groups.
Functional groups serve many purposes. First, they divide the test cases of a test plan into smaller sets. Second, they provide some basic "shape" for your test plan. By creating test case groups you specify the general aspects of the software you want to test.
Some examples of functional groups are:
You can create you own groups, or use the predefined templates supplied with BTR
After you create functional groups for your product, you are ready to start creating test cases.
A test case can be defined as the set of operations that exercise some functionality in your product, a set of inputs, and the expected results. A test case may or may not be automated. BTR considers the following elements in a test case:
The summary, action and effect fields are stored and displayed in HTML. Sometimes a test case can be pretty complex and HTML elements such as tables or lists can better represent the information.
You can add and edit your test cases with the Web GUI provided by BTR, or create an HTML file and import it into test cases later on.
Test cases may also share information. Imagine, you have a set of test cases, which require the same initial conditions, e.g. "modem disconnected from the phone line". You can repeat that sentence in the Action field for every test case. This sounds redundant and it really is. Instead, you can use tags. A tag is something that can be attached to more than one test case and helps you to avoid repeating the same entries over and over again.
Going back to our modem example, you can define a tag named "modem disconnected" and add it to all the test cases that need it.
A tag has a name and a value. The name is limited to 240 characters, while its value can be much longer. Some tags may consist of a name only, like in the modem example.
Note that what you use a tag for is up to you. A tag can define conditions that are the same for a number of test cases. It can also be treated as an additional attribute of a test case.
Unlike the summary, action and effect fields, test case tags are only linked to a test case, not to its log entries. On this way, if you run a test case and then change its tags, you will see the current tags on its log entries.
Each test plan has its own list of tags.
BTR allows you to define not only what to do on each test case, but also who should do (run) it. This is what the tester list is for.When you run a test plan, BTR creates copies of each test case for each defined tester. By default, all test cases are assigned to all testers. You can change this by assigning testers to components.
When you are done defining your test plan document, creating your test cases and assigning testers to components, you are ready to run the test plan.
BTR distinguishes between test plan and test plan run. A test plan is defined once and may be run multiple times. As your products change, their test plans change too. Some test cases are added, some deleted, many are changed.
A test plan run on the other hand, refers to one specific version of a product and test plan. When you run the test plan, test case log entries are created for every defined tester. Testers check test cases one by one marking its log entry as "passed" or "failed." Testers have their own separate log entries.
When a test case fails, we have a bug, and here Bugzilla is in charge. You can invoke the Bugzilla "new bug" form right from the BTR test case log form. Reference to the bug is stored with the test cases log entry.
You may run the same test case many times. Later on, even if you change your test plan (its cases or the plan document), your test plan runs keep referring consistently to the proper description of your test cases, as they were when you run them.