While it might hurt to admit it, you are not perfect. That is why we perform website testing, or software testing.
Testing is not random. No matter what type of testing you are performing – a future post perhaps – you always have a set of expected behaviours or appearances that you expect for your project.
This is where a testing template can come in handy. A testing template allows you to pragmatically think through what you would expect as key acceptance criteria for the test to pass, and why.
There is no fixed way to perform testing, however the considerations which need to be made are common to all tests, in varying degrees. So, based on let’s break this down.
What are we testing?
Who we are testing for, and why we are testing, can theoretically be summed up into a user story:
As a [user], when I [perform an action], then [the result you expect to see].
Typically this would be accompanied by a set of acceptance criteria in order to flesh out further what results you expect. On an eCommerce site for example, one might have the following:
As a customer, when I visit the company homepage, I expect to be presented with a selection of items to be able to purchase.
- Presented with a selection of recommended or featured products
- All images on the page load
- Page matches the sample file provided by the design team
This is just one example of a test which may be performed. There are myriad, from ensuring the website loads, to ensuring the customer is correctly added to a mailing list.
The budget does not necessarily refer to a fiscal budget, but also to an hours or other resource budget. Within development, assuming a testing strategy is in place utilising some tools such as GhostInspector would be part of the operational budget, as would salaries, so it would be more in line with allocating staffing hours.
How critical a particular test is can also be a factor here. If we are simply checking to ensure adherence to a design specification across browsers, this will result in a substantially different resource and testing budget to an auto-pilot system within a plane.
As above, timeline here can mean multiple different things. Primarily, timeline defines how long testing is planned to take. Other factors which may be considered here would be recurrence.
You may wish to perform a set of automated tests every time you build a new environment for a pull-request, and again after merging into master. You may need to perform it daily to ensure there are no issues with server performance. Each of these factors should be considered within a testing plan.
The scope of a testing plan is arguably the most important part of the testing plan overview. This is where you state to what extent you test the website across different devices, browsers, or operating systems.
The scope for testing can be split into 3 or 4 ‘support levels’. These outline the performance criteria expected. At support level 1, the website is expected to perform exactly as specified. Any deviance from the acceptance criteria is not tolerated, and the website will be passed back for further development.
At support level 2, some deterioration of performance is acceptable, so long as the user gets a mostly complete user experience. Business critical elements are required.
No support is offered at level 3. As a result, the website may or may not load correctly. Users should upgrade their hardware and/or software. Ultimately, the concept of these support levels is that the website, or software, degrades gracefully as it encounters older and more outdated technologies.
The reports generated from testing will vary by the testing scope, however each will report based on the different support-levels and acceptance criteria. If testing is being completed on a whole site, performance bench marking can also form part of a test. Tools such as Google’s Lighthouse are a great example of this.
Testing is critical, but often overlooked. The adage: failing to prepare is preparing to fail.
By not planning testing of projects correctly, you are potentially exposing yourself to the risk of failure. As such, deciding to what extent testing will occur will be directly derived from how business critical the software is.