A test plan in Bugzilla Test Runner (BTR) consists of the following elements:
Every test plan has its test plan editor. The test plan editor manages the test plan and have special rights over its test cases, test runs, and test case logs. The editor is a valid user account in Bugzilla.
A test plan is also described by its type. The list of available test plan types is defined in the database and can by customized through SQL. See "Customizing the BTR database" section in this manual for details.
Access to test plan documents and test cases is controlled by two special Bugzilla groups:
Access to these groups can be set through the "Edit Users" Bugzilla's page.
You don't need special privileges just to view test plan documents or test cases, provided that you have access to the Bugzilla's product they are related to.
A test plan document specifies the outline of your test plan, providing guidance to the testers that will perform the test cases defined as part of the plan. Generally, you put here everything that is not related a particular test case. BTR does not provide any structure for this document. What you put here is up to you.
The test plan document is stored as HTML. Tables, hyper-links, lists, etc. can be used freely to create a good document.
BTR provides a WWW interface to edit your test plan document. You can however use any editor of your preference. BTR can import any HTML file as a test plan document. The way this import works is very simple - everything between the <body> and </body> tags gets imported. You should keep one thing in mind: some HTML-editing packages can fill your document with lots of stuff that you actually don't need, e.g. font specifications for every paragraph, etc. It is a good practice to have a look at the source code of your HTML document before importing it into BTR.
Every time you change anything within your test plan document, the old version is archived in BTR's database. BTR can show you the full history of changes.
The list of test cases is probably the most important part of your test plan. In BTR, a test case consists of the following attributes:
A test case can be assigned user-defined attributes using tags. A tag consists of a name and a description. It can be treated as an additional attribute of a test case. Tags have also one very important feature: the same tag can be assigned to more then one test case. This way, you can use tags for common attributes.
Test case tags are an universal mechanism, and can be used in number of ways. One common way is to use a tag as an attribute. For example: a tag can specify initial conditions for a set of test cases.
Tags may also be used in grouping test cases. You can, for example, use tags to indicate a test case priority, hardware requirements, etc.
Test cases are divided into functional groups. This is intended to help you to organize your test cases. Specifying functional groups, you give a basic layout to your test plan. It is easier to figure out test cases when you have a decent set of functional groups defined.
So far, we have defined what should be done to perform test cases. Now, we are about to specify who should perform them. When you create a test plan, you assign a number of users to the test plan as testers. Users here are Bugzilla accounts.
You not only assign testers to test plans. You can assign them to components as well. This way you can partition a set of test cases among testers, limitting the number of test cases each tester is to complete. So far, component-based selection is the only way to assign a subset of test cases to individual testers.
A test case log contains information regarding the execution status of a particular test case. By default, a tester cannot even see the status of other testers' logs. This is to help the testers resist the temptation of copying others' work instead of running all their assigned tests on their own. If you do not require this limitation, you can turn it off. See the parameters section on how to do it.
So, for a given test case log, only the tester and test plan editor can watch its status. You can change this using the watchers feature. Watchers are privileged users allowed to see test case logs without those limits. Any valid Bugzilla user account can be set as a test plan watcher.
Although BTR has a web interface to add test cases, it may not be the best way to enter a great number of test cases. This is especially true when you already have your Test Plans defined and want to add them in BTR. That's why BTR supports mass-import of test cases into its database.
To export test cases to an HTML document, click export on your test plan form. Test cases are printed in a format compatible with the import file format.
The import file must be an HTML document with a special structure. This structure is intended to be both suitable for parsing and human-readable. You can either add or modify test cases through the import file.
See the example_cases.html file for an import file example. You may import these test cases into your testplan, provided that you have a TestProduct and a TestComponent defined as a valid Bugzilla's product and component, respectively.
BTR uses HTML tags to define keyword-elements. An import file may contain definitions for the following items:
Notice that this doesn't include the test plan document. See the section Test Plan Document for information about how to import a test plan document.
The list of test cases in an import file must be preceded with the declaration of a product, a component and the functional groups to which the test cases belong.
Here is a listing of all the special statements used in an import document:
<H1>Product: product_name</H1>
<H2>Add group: group_name</H2>
<H2>Change group group_id [group_name]</H2>
<H2>Add tag: tag_name</H2>
<H2>Change tag tag_id [tag_name]</H2>
<H2>Component: component_name</H2>
<H3>Group: group_name</H3>
<H4>Add case: [confirmed|disabled|proposed]</H4>
<H4>Change case: case_id [confirmed|disabled|proposed][,[manual|automated]]</H4>
<H5>Tag: tag_name</H5>
<H5>Requirement: requirement</H5>
<H5>Summary:</H5>
<H5>Action:</H5>
<H5>Effect:</H5>
<H5>End</H5>
As one can guess, Add ... keywords define new elements to be created, while Change ... keywords change existing elements. In this last case the ... id parameter is used to identify the element to change.
<H1>Product: product_name</H1>
This statement declares the product for all the elements that follow. This should be the first special statement in your import document. It can be preceded by any text, which will be ignored.
<H2>Add group: group_name</H2>
This statement adds a new functional group to your test plan. Everything that follows Add group: up to </H2> is considered the name of the group. Leading and terminating blanks are ignored.
Any text that follows this statement is considered the text of the new group, up to the next special statement found.
<H2>Change group group_id [group_name]</H2>
This statement changes the definition of an existing functional group. The mandatory group_id parameter (a decimal number) must be the id of a valid existing group.
The group_name parameter is optional. If used, it defines a new name for the group.
<H2>Add tag: tag_name</H2>
This statement adds a new tag to your test plan. Everything that follows Add tag: up to </H2> is considered the name of the tag. Leading and terminating blanks are ignored.
Any text that follows this statement is considered the text of the new tag, up to the next special statement found.
<H2>Change tag tag_id [tag_name]</H2>
This statement changes the definition of an existing tag. The mandatory tag_id parameter (a decimal number) must be the id of a valid existing tag.
The tag_name parameter is optional. If used, it defines the new name for the tag.
<H2>Component: component_name</H2>
This statement declares the component for the test cases that follow.
Test cases defined after this statement will be assigned to this
component, up to the next Component: statement.
<H3>Group: group_name</H3>
This statement declares the functional group for the test cases
that follow. Test cases defined after this statement will be
assigned to this group, up to the next Group: statement.
<H4>Add case [confirmed|disabled|proposed][,manual|automated]</H4>
This statement adds a new test case to the test plan. The test case is assigned to the product, component and group defined previously by the Product, Component and Group statements. Optionally, you can specify the status and execution mode of the test case. If not specified, the status is set to confirmed and the execution mode to manual.
The principal attributes of a test case (summary, action and effect) are defined by corresponding special <H5> statements. None of these statement are mandatory - you can define an "empty" test case if you like.
Test case attriutes are defined by the following special statements:
The test case definition is terminated either by <H5>End</H5> or Add/Change case statements.
<H4>Change case case_id [confirmed|disabled|proposed][,[manual|automated]]</H4>
This statement changes an existing test case. The test case is identified by its case_id. Optionally, you can specify its new status and execution mode.
You can change a test case attributes by using the Tag, Requirement, Summary, Action and Effect statements. Only specified attributes are changed.
<H5>Requirement: requirement_name</H5>
This statement sets the requirement attribute of the current test case.
<H5>Tag: tag_name</H5>
The Tag statement applies a tag to the current test case. You may include many Tag statements in your test case definition.
Warning: notice that any tag defined inside a Change case statement first clears the list of tags associated with the test case.
<H5>Summary:</H5>
This statement specifies the summary of the current test case. This is usually a short text that identifies the test case. It may also be a code name which follows your organization conventions.
You may specify the test case summary implicitly, without using the Summary statement: anything that you place between the Add/Change case statement and the next special statement (Action, Effect or End) is considered the summary.
<H5>Action:</H5>
This statement specifies the test case action: the set of steps to be done to perform the test. The action text starts immediately after the Action statement, up to the Effect or End statements.
<H5>Effect:</H5>
This statement specifies the test case effect: the set of expected results for each step in the test case action attribute. The effect text starts immediately after the Effect statement, up to the End statement or a new test case definition.
<H5>End</H5>
This statement is an optional way of terminating a test case definition. If you don't use it, the test case definition is automatically terminated by the next Add/Change case, Group or Component statements.
The End statement can be helpful if you want to add special separators between test cases in your import file.