Quality is not only an output; it’s an outcome over the output

You may think what the differences between these two are, they sound synonymous. It’s true that they are, but there are some differences.

Let’s first talk with the definition of these twos.  In my sense, in the non-profit world, outputs are programs, training, and workshops; outcomes are knowledge transferred and behaviors changed. In the material world, the distinctions are not always so clear. Let's define outputs as the stuff we produce, it may be physical or virtual, for a specific type of customer—say, Foot over-bridge for the people. And let's define outcomes as the difference that it makes—keeping the people safe from the road accidents.

Outcomes are the benefit your customers receive from your stuff. This starts with truly understanding your customers' needs—their challenges, issues, constraints, priorities—by walking in their shoes and in their neighborhoods, businesses, and cultures. See what's inconvenient, taking a lot of time, money, and/or effort.

Outputs are important products, services, profits, and revenues. Outcomes create meanings, relationships, and differences. Outputs, such as revenue and profit, enable us to fund outcomes; but without outcomes, there is no need for outputs. 

Considering these definitions and differences, let’s come to the world of software and services. Here, the product itself i.e. any software or service is an output. And, the needs that have been fulfilled by it are the outcomes. So, what the users or audiences look forward to it are their needs and wishes, not actually the product itself. So, if the product cannot fulfill their needs and wishes, it is not the outcome. It is only an output with no uses. And, also if the product can have the features to fulfill the needs, but somehow the users or audiences cannot use it for their needs, it is not the outcome as well. Here comes the use of the term ‘Usability’. If a product or service is not usable enough to get the work done although there is major fault in its functionality, then there is no outcome. So, usability is an important issue to let the user finding the outcomes. So, as QA or Test engineers, it will be our responsibility to look for fine-tuning the outcomes along with the output instead of looking for the output only.

Discovering Heuristics

At first, I would like to convey my special thanks to James Bach for the idea and inspiration to write this blog post about my own heuristics which I follow while testing. Actually, I was not aware of the heuristics that I follow unconsciously while testing; James helped me to find out these.

Initially, I would like to discuss briefly about Heuristics and then I will explain what approaches I usually follow in my own work.

Heuristic refers to experience-based techniques for problem solving, learning, and discovery that gives a solution which is not guaranteed to be optimal. Examples of this method include using a rule of thumb, an educated guess, an intuitive judgment, stereotyping, or common sense. [Wikipedia]

[A heuristic is a fallible method of solving a problem or making a decision : James Bach] According to James, Heuristics are not experience-based, they can be theory-based. 

Applying heuristics needs two things generally. First, we have to learn about heuristics. Being Knowledgeable about heuristics gives us the idea to apply and encourages us to develop our skills.  Second, we should understand when a heuristic or method is helpful, when it should be used and when it should not. Moreover, we should learn how to frame our own problem so we can apply the best heuristic to solve the problem.

To develop the heuristics, we have to continuously look for ideas in testing that frequently get us the results. Perhaps, some ideas that often help us to uncover the defects in our own way. For that, we have to look for the patterns or commonality that we follow consciously or unconsciously while testing a product.

Now, let’s talk about what I have found when I tried to find out my own heuristics:

Most of the time at the beginning of testing any product, what I do is without looking into any specification/requirements/document, I just look into the product with a blank mind to find out what is the product actually. In my sense, this helps me to remove any kind of biases which I might have.  Sometimes it is helpful and sometimes it is not. But, it helps me to look into any product with the fresh eyes. And, fresh eyes without any bias helps to generate so many questions and which results in broadening the ideas/scenarios for the particular product.

So, in my case, playing with the product without knowing about it may be a heuristic. May be we can term it as 'Ignorant Heuristic'.

Another testing strategy that I usually follow is, after testing a product/feature for a certain period of time , I do give a pause on the testing and go to other things (May be testing other product or anything that is not related to the product itself) to give my mind a break so that it can start thinking again from the beginning. Then, I again come back to testing of that product. (Here, this ‘Certain Time’ is totally dependent on the context and my mind. May be, I can come up with another heuristic which may tell how I can decide on this ‘Certain Time’)

Most of the times I have found that, while doing the other things in the break, some thoughts which were not connected while I was doing the test, get connected unconsciously in my mind and help me to find a better scenario. May be, that is how my mind works. It might need a break to connect the unconscious ideas together which I experienced during the testing. Most of the times, I found that, there are some simple ideas/concepts which I missed or couldn't grasp well at the time, but can reconnect them on the breaks. Sometimes, there are some critical ideas as well which I can reconnect at the breaks.

So, giving your mind a break may be a behavioral skill/heuristic to follow while testing something. In my case, it works. May be, we can term it as "Kit-Kat Heuristic" (From the tagline of Kit-Kat: Have a break, Have a Kit-Kat)

There may be many more heuristics like the above two for me and there will be a lot for you as well. The thing is, we need to find out these from our own way of doing things, so that we can use them wisely and effectively in our work.

[Courtesy: 1. Wikipedia
                 2. James Bach ]