Dan Ashby
Director of Quality Services
Proud father.
Quality and Testing enthusiast, teacher, coach, and mentor.
Director of Quality Services @ Ada Health.
I blog at: danashby.co.uk
I love being involved in the testing community!
Badges
Contributions
What's a bug report?Firstly, a bug, commonly called a defect or problem, is anything that is perceived to threaten the value of the software. A bug report, is the information about the bugs that we find that we collate and supply to the product owner, stakeholders, or developers. Some means of documenting and communicating our defects within the application that we're testing or the software.
Can you give some examples of things that you'll find in bug reports?As part of bug reports, we will have, usually, a a summary of what the bug is, a detailed description with some environment information, attachments of screenshots or videos of the bug, and log files possibly too. And you would also have a priority and severity for the bug.
What's the value of a bug report?Logging a bug allows us to track the progress of that bug. It also may be some time before the bug is addressed depending on its severity and priority. So logging the bug means that we won't forget about it. And it's a good starting point for developers to be able to fix an issue.
What are the pitfalls?Counting bug reports as a means to determine quality can be misleading. Quality is subjective, and one person's bug is another person's feature. And also just focusing on writing a bug report can have adverse effects on having the collaborative and meaningful conversations about the bug by developers and stakeholders as well. And also badly written bug reports can also damage your reputation as a tester.
What is exploratory testing? (ET) Exploratory testing is an approach towards testing. It involves concurrently learning, designing, and executing, and recording tests, at the same time rather than specifying all your tests upfront before execution, and it has the purpose of uncovering information related to risks. Do you have any examples? I could do things like explore different types of data that could be submitted via a web form or explore the rendering of a web page across multiple browsers or explore how a feature works depending on different user privileges. What's the value of utilizing an exploratory testing approach? Exploratory testing empowers, testers to use their skills to discover information freely. ET can be structured to help keep the tester focused on specific risks, and it can help, a tester to reveal more information that a scripted test approach wouldn't uncover as you're not biased by the test case steps. Are there any pitfalls? Exploratory testing is an approach that requires a lot of skill and practice. A good exploratory tester needs to know how to focus and defocus when testing, And note taking is important to help communicate what a tester has done whilst not hampering them, but this is hard to do.
Why is influencing relevant to testing?So influence is the skill to affect something to your ideal. So influence skills are important for a tester as the topic of testing is generally misunderstood within other areas of software development. So you need to be able to influence people in order to be able to teach them about good testing practices.
Do you have any examples? So the rule of reciprocation is a great influence technique where if you do a positive favor for someone then they will be subconsciously obliged to return the favor. You've got rejection and retreat which is an influence technique that's made possible by the contrast principle where if you feel an idea that you want to pitch will be flat out rejected, pitch another idea that seems less appealing so that it gets rejected. And the second idea, the real idea you want to pitch, seems much more appealing.
What's the value?So influence skills can definitely help you overcome difficult situations as a tester. Many people have many misunderstandings of testing such as when you hear things like automate all the testing or testing is just checking requirements. Influencing techniques can help open the door to communicate to those people to teach them about what testing is and what good testing looks like.
Are there any pitfalls?Influence techniques can sometimes be confused with negative manipulation activities. Some people confuse things like bribery and guilt tripping as influence, but they're not. It's important to remember that sometimes influence techniques don't work, and it's also important to remember that you may actually be trying to influence incorrect messages based on your own misconceptions. So be aware of your own biases in the ideas that you're trying to promote.
What is session based test management?Session based test management or SBTM and it's a method for measuring and managing exploratory testing. It's a management tool to help others see how a tester is progressing in their exploratory testing work, and it empowers testers to store test notes and useful metrics for other team members to use without sacrificing the quality of the testing.
And how does it work?A tester will create a charter and add an estimate of length to it, small, medium, or large. The tester executes the charter as an exploratory test session, and once complete they'll fill in a report that contains test notes, the actual time spent, and metrics on the percentage of time spent on test design and execution, bug investigation and reporting, and session setup. And then at the end the tester debriefs to another tester to ensure that they're happy with what they've tested and reported.
What's the value of SBTM? The metrics can reveal how a tester is getting on. For example, how much time they're actually testing versus setting up an environment, and charters can be calculated with estimated lengths to give an estimate of time to complete testing.
Any pitfalls? You and your team need to be disciplined when it comes to running debriefs. Missing them could affect the quality of the reports. Also, SBTM is not a widely accepted or known approach, so it requires some effort towards selling it to to stakeholders or managers.
What are acceptance criteria?Acceptance criteria are a list of expectations that a user will have, when using a feature. They are written in short sentences to define a specific behavior of a feature, and they are commonly added to user stories to add more description.
Do you have any examples?Examples could be, the widget should have a text box and character count. Entering characters to reduce the character count could be another example. Or the third one is when character count is at zero, no more characters can be added to the text box.
What's the value?It gives the team a boundary on what they should and shouldn't be delivering in a feature, and acceptance criteria, can help with discussion and collaboration of features. And acceptance criteria can help testers generate test ideas and identify risks.
Are there any pitfalls?Acceptance criteria can help stem testing ideas, but they are not tests themselves. And vague acceptance criteria can potentially lead to misunderstandings, and bugs in the system too.
What are user stories? User stories are a tool that are used by, agile teams to capture a description of a feature. They follow a template of as a I want so that, and they are typically put on a card or stored in a tool like Jira.
Do you have any examples?An example could be, as a user, I want to change my email address so that I can keep receiving notifications to my new email address. Or as an admin, I want to edit records so that I can keep my records up to date.
What's the value?They help facilitate discussions about feature within your team. They can be used to track progress of the development of a feature, and, it helps teams determine their workload.
Are there any pitfalls? It's important that your user stories have clear value to a user. Vague user stories make it harder to develop and test.
What are software testing notes?Software testing notes are used for recording information during different testing activities, whether it's testing an idea or a product. And testing notes can come in many forms. Some are more suitable for certain activities than others. And notes can be collated for future re reporting, or sharing information with your team.
Do you have any examples of software testing notes?Mind maps with multiple nodes to demonstrate different ways you have tested the product, and to show more information about your testing, test ideas and discoveries and problems, or diagrams to demonstrate behavior during the requirements of planning sessions is a good example, or tools like RapidReporter or JiraCapture, to notate your exploratory testing in a chronological text based format.
What's the value?Good testing notes can stem more test ideas. They can help you track different activities beyond executing tests, such as session setup and bug investigation. They can also be used as a trigger to remind you of what testing you have done during an exploratory testing session. They can be used as testing evidence displaying coverage and risks that have been investigated.
What are the pitfalls?Writing too much detail in your testing notes can be a drag on your time and may affect what information you're taking in. And writing vague notes with too little detail will result in a lack of understanding in the future. And sticking with one note taking approach may bias the way that you test to so experiment with different note taking approaches.
What is an oracle?An oracle in testing is a source of knowledge that informs your testing, and that information can be used to inform or determine your testing or to determine if something is a defect or not. Oracles can sometimes inform you of other oracles, and oracles are like a spidey sense when you get the feeling that something doesn't quite seem right here. Cool.
Do you have any examples?
You have comparable products. So does the application have to work in a way that is in line with similar products?
Statutes. Does the application have to work in a way that conforms to laws and regulations.
Claims, does the application have to work according to requirements or specifications. Are there any marketing or sales claims made regarding the software, or past experiences, so relating to things that you have experienced in the past to inform you of the software that you're testing?
What's the value? They can be used as a source, for backing up your book reports, and, also, they can inform your test design and what might take priority in your testing.
What are the pitfalls?Oracles are flawed. Using one oracle will return only one piece of the puzzle. You need to use many oracles to gain a fuller picture. And now we know about oracles.
What is technical testing?Technical testing is an approach where testers use technical skills such as programming, scripting, and development practices to inform their testing. This approach can be used to help testers reveal in-depth information of a product that typically less technical based testing would not reveal. Technical testing is more of a mindset than a set of prescribed activities, focusing on how technical knowledge can help me as a tester.
What technical testing examples are there?Using browser tools to learn technical details about my product and manipulate my product in unusual ways Pairing with a developer as they build the product to test their ideas and the code as it's created, I could use a HTTP proxy tool to monitor traffic between product components.
What's the value?The technical knowledge of your product can help you generate new test ideas and inform your test strategy. Technical knowledge can help communicate with developers by providing detailed evidence of bugs or having technical conversations about new features. You can use tools to create complex tests to discover interesting and complex bugs that sometimes have severe consequences.
Are there any pitfalls?Whilst understanding how to code can help you with technical testing, Technical testing is not solely about coding. There are tools that can be used that don't require programming skills. Also, focusing too much on the technical activities in testing can become a distraction with testers wasting time on setting up or building tools.
What is scripted testing?Scripted testing is an approach to testing that is aimed at confirming that a product meets expectations set from requirements. Scripted testing typically uses test cases to aid the comparison between the expectations and the product itself. Scripted testing is sometimes referred to as checking to emphasize that scripted testing does not cover investigative testing activities. And scripted testing is usually reported in a quantitative manner following the pass or fail output of whether the expectation is met or not.
Got any examples?So an example of a test case could be step one, browse to the search screen. Step two, add search criteria. Step three, click on the search button. The expected result is that the relevant search results relating to the search criteria should be displayed in the results table, and the table should be paginated to ten results per page.
What's the value of scripted testing?So scripted testing could be a good starting point to follow steps to help understand how to operate the software and how it's expected to operate. Scripted testing can easily be repeated for change detection. If a previously passed expectation is now being met, we can assume something has changed. Scripted testing can also stem other testing activities that complement scripted testing in line with the whole testing picture.
What pitfalls exist with scripted testing? So an overreliance on scripted testing means that you will miss potential bugs and information. Scripted testing is also expensive. It takes time to create and maintain scripted tests, and scripted testing alone does not constitute the whole of testing. There are many misconceptions about this.
What is lateral and critical thinking?Lateral thinking is where we try to think creatively about an idea, requirement, or piece of information from different angles to uncover more information about variables and perspectives relating to that initial information. And critical thinking is where we critically assess the information that we have, thinking about risks, assumptions, and problems relating to that information that we are assessing.
Do you have any examples? So thinking laterally about dice, it can have between four and a hundred and twenty sides. The number of the dice can be displayed as dots, digits, or even as written text on the dice. The dice can be made from plastic, wood, or many different kinds of metal. And thinking critically about the fact that we need the dice to play the game, we can ask questions like, well, why do we need the dice? Would our random number generator, be suitable instead?
And what's the value?So lateral and critical thinking skills help us to uncover test ideas. If we are testing the idea of the software early in the SDLC process, then our lateral and critical thinking skills will help us dispel assumptions and uncover more information to help us prevent problems and risks further down the line.
And what are the pitfalls?So many people aren't familiar with lateral and critical thinking and don't realize the importance of these skills for testers. And many companies don't fully understand the investigative side of testing. And even with strong lateral and critical thinking skills, it's important to remember, you can't think of every permutation or each variable.
What is testability? Testability is a measure of how easy it is for testing to happen in a given context. Testibility is a measure of many drivers such as people, product, vision, process, time, resources, and problem. These drivers can have a positive effect on our testing, making it easier to test, or they can have a negative effect to making it more difficult to test. Interesting.
Do you have any examples?The team's attitudes toward testing can either mean testing is prioritised and supported, or it's an afterthought. A product with many accessible layers such as APIs means we can be more targeted with our testing and automation. Access to testing tools could be restricted due to budget or IT policies. This impacts the types of testing we'll be able to do.
What's the value of knowing about testability?Knowing about testability informs our testing approaches and our strategies. It provides clear framing for the testing we do and don't do, which helps communicate that work to others, improving testability often delivers more value to the team than doing more testing activities.
And what are some of the pitfalls around testability?Testability is fluid. Therefore, teams need to periodically review it to ensure approaches are fit for purpose. As testability is an assessment of contact drivers, it can often be biased by those doing the measuring. So it's important to include the whole team.