Once a test has been launched (or scheduled to launch later), you can see its results in the Tests results list page, accessible from the left side menu:
Tests results menu link
Tests results list page
All past and current tests results are here, continuously updated:
Tests results list page
The results are paginated, and you can filter them by creator, platform, test or tags. You can also search them by name.
Results actions
You can easily rename a result if you want to, by clicking its name:
Result renaming
Or by clicking the three dots button ⋮ at its right:
Delete and rename options
You can also add or edit tags on each tests result:
Result tags
All of these options are here to help you better organize your results.
If you click Add on a tests result, it will be added to the list in the right part of the page:
Run again
Then, you can trigger a re-run if you click Run. You can also generate an excel report (in the XLSX format) of all your selected results, by clicking the arrow ▼ button and then the Generate Excel report button:
Generate report
Test states
A test may go through different states before completing:
-
Scheduled: the test has been scheduled to run at a later time. It is not started yet.
A scheduled test
-
Pending: the test is ready to start but waiting for all the resources it needs to be available. Some of them may be used by another test, which makes them temporarily unavailable.
A pending test
-
Ongoing: the test has started and is currently running.
An ongoing test
-
Sleeping: the test has been paused for a determined amount of time, because of a pause action.
A paused test
-
Canceled: the user manually canceled the test.
A canceled test
-
Success: the test completed with no error.
A successful test
-
Failed: an error occurred (either an action could not be played, or an assertion failed).
A failed test
Detailed tests results
If you click tests results in the Tests results list page, you will be taken to the results details page:
Results details page
This page contains a lot of information. At the top is the summary of the run: number of successful and failed tests, date and time of the run, the platform the tests were run on, and the user that triggered the run. In the top right corner are the To Excel button and Run Again buttons, respectively used to export the results to an XLSX file, and trigger a re-run (as in the Tests results list page).
Below are the details of each test with the results of each step in chronological order. Keep in mind that the presentation may be different based on what kind of actions your own tests contain. Each action has a different presentation. For instance, for an API call, you will see the request and response headers and body, for a smartphone script, you will see each step inside the script along with a screenshot, etc.
Collapsible parts
Since this information may take a lot of vertical space, there are many parts that you can collapse by clicking them:
Collapsible areas
- The whole test area is collapsible.
Collapsed test
- The action command area is collapsed by default, and expandable if you want to see its details. Here is what is looks like while expanded, in the case of an API call action:
Expanded action command
- The details of an action result are usually collapsible, depending on the kind of action. This is the case of the API call, since the response body can be quite long if the response is big, as in our example:
Collapsed API response
Common results information
While each action result has a different presentation and different information to display based on the action type, there are several areas that will always be present:
Common areas
- The step summary: step index, action type, failure or success state, timing.
- The action command.
- The assertions results block.
- The variables block, showing all the variables used, created and/or changed during the step.
Step summary
The step summary has some hidden capabilities: you can click the step's timing to display timing details (the start and end time):
Timing details
Last but not least, if you click the three dots button ⋮ below the timing, you will find a menu with test replay options:
Replay options
These buttons let you trigger a re-run in three different ways:
-
Replay all from this step: replay all the tests (not only the one that the step belongs to, but also the next ones), starting from this step.
-
Replay until the end of this test: replay the test that this step belongs to, starting from this step.
-
Replay this step only: replay only this specific step.
Action command
This block shows the configuration of the action (e.g. for a ping, which address was ping-ed, for an USSD action, which USSD code was sent, etc). If any variable was used in the command, its value is showed instead of the variable.
However, you will notice that if you hover over the step area, a button appears in the top-right corner. This button Display variables / Hide variables lets you choose whether to show the variables values (default mode) or the original variables names:
Toggle step variables
USSD / SMS message update feature
In the case of Wait SMS action, USSD Command action and Wait USSD Push action, a special button in the result block allows you to change the expected SMS / USSD message:
Update button
This function comes in handy if you have several tests that use the same SMS or USSD expected messages, because it lets you update the message across all the tests at once, for both SMS and USSD actions:
Update message page