Test scenario
The test scenario is the top-level element of testing in Wenenu. Test scenarios group together test steps that are executed sequentially. A test scenario can be executed manually or it can be scheduled for execution at regular intervals. Wenenu creates a comprehensive run report for every scenario execution.
|
Users need Wenenu Owner or Wenenu Contributor roles to create, modify, execute or schedule test scenarios. Users with the Wenenu Reader role can see the result of scenario executions. |
To create a test scenario, users need to give a name and the report time zone. Name is an arbitrary string up to 256 characters. The report time zone is used when PDF run reports are created. In the report, dates and times are displayed in the given time zone.
After creation the test scenario is empty, it does not contain test steps. The scenario configuration can be changed on the scenario edit page. Users can add up to 200 test steps to a scenario. To be executable, a test scenario should contain at least one test step. On the scenario edit page, the list of test steps is on the left and the settings of the selected step are on the right.
Variables
Variables are name-value pairs attached to a test scenario. Variables are global to the test scenario. A test scenario can have up to 200 variables. The name of a variable is an alphanumeric string up to 64 characters that can also contain hyphens and underscores. It must start with a letter. The value of the variable is an arbitrary string up to 5120 characters.
|
|
Regular expression of the name of a variable is ^[a-zA-Z][a-zA-Z0-9_-]*$
|
Variables can be used in the settings of the test steps. Generally, variables can be used in step configurations where arbitrary values can be set such as hostnames, database names, usernames, passwords and so on. Some exceptions are mentioned in the documentation of the individual test steps. Variable usage syntax is ${variable-name}
Variables can be made secret by clicking the lock icon next to the value field of the variable. Secret variables are useful to store passwords and keys. The secret values can be used in step configurations without displaying them as plain text. Once a secret variable is set, its value is never returned to the users.
Variables are also usable in code execution test steps. The variables are passed to the program as environment variables. The way to reach the environment variables is specific to the programming language of the test step.
Scenario details
The scenario details page displays information about the previous executions of the test scenario. There are three tabs on the page Graphs, Previous runs and Run details. The Graphs tab shows information about the last runs of the test scenario in the form of graphs.
The last ten previous scenario executions are displayed on the first graph. The bar chart shows the date, state and run time of the scenario runs. Hovering the mouse over a bar on the graph reveals additional information about the start and end times and the completed test steps of the scenario execution.
If the test scenario successfully started Azure Site Recovery recovery plans, the statistics of the recovery plan executions are displayed on the subsequent graphs. The graphs show the number of protected items, the recovery time actual (RTA) and the average recovery point actual (RPA) values. Recovery time actual is the time that elapsed between Wenenu calling the Azure ARM API to start the recovery plan and the time when the recovery plan finished starting. The average recovery point actual is the average of the time differences between Wenenu calling the Azure ARM API and the recovery point times that were used for the virtual machines in the recovery plan.
Previous runs
The previous runs tab shows the previous executions of the test scenario. The table shows the scheduled time, the actual start and end times, the state and the test step count of the scenario executions. The finished failed and skipped test steps are displayed next to the test step count in brackets. Once a run finished, the execution report can be downloaded by opening the context menu and clicking on Download report. Clicking on a scenario execution switches to the Run details tab and loads the steps of the selected run.
Run details
The run details tab shows the test steps of the selected run in the order of execution. Display name, state, step type and start and end times are listed for every test step. Clicking on a test step displays its results in plain text format. While the test scenario is still running, the run details tab is refreshed periodically.
Scheduling
Scheduling can be enabled on test scenarios to execute them at regular intervals. Scheduling requires a time zone a frequency and the date and time settings according to the selected frequency. When scheduling is enabled, the test scenario will be executed according to the defined frequency and time in the given timezone.
Reporting
Wenenu creates a comprehensive PDF report for every test scenario run. The report can be downloaded on the previous runs tab of the scenario details page for the individual scenario runs. Email reporting can also be enabled on the test scenarios. Wenenu sends out the PDF report after the scenario run is finished to up to three email addresses. The timezone of the created report can be changed on the email report modal window. The new timezone setting will affect only the PDF reports that are created after the change.
