On The Lighter Side

Testers have a life too. Oh yeah, we pretty much do. Its usually the developer who spends sleepless nights. Dont believe me? I have someone to back me up.

I have been asked what else do I do besides testing stuff. First off, let me tell you that testing is more of a hobby right now. Its something I am passionate about but its not what I am into full-time. Where I presently work, I am not into full time testing. As a matter of fact we work on clinical trials. Work on creating programs which would generate certain statistics for a particular drug being developed. Yes, its very different from the normal software world. Why am I doing it? Just because its something new I am getting to learn. It gives me scope to draw an analogy of the pros and cons of the software/IT world compared to the pharmaceutical world. And it throws light on some different aspects of design and testing.

What else do I do normally? Well, most of the time you could find me behind my laptop. Either its some new thing I am trying out (you can spend a whole day trying to figure out inheritance even after you thought you knew it well - I'll see if I could post some examples later). Sometime I am looking up whats happening on Mozilla. Lately I am trying to look deeper into algorithms and data structures. Make my brain work that little bit more.

Then there is walking/jogging. It sort of depends on my mood. When there is a lot of thought process going on, I prefer long walks. It helps you clear out things in your head and get fresh in no time. Else I'd rather run on the treadmill. You'll probably find me at Bally's every alternate evening.

Cooking. yeah. I enjoy cooking. But I wont go more into that. But the day I open my own chain of cuisine, everyone shall be invited ;-) And then, there is music. I do play the key board but its been years since I touched one!

Why is this post a little off topic? Just to let you know its important to strike a balance. Companies love people who are dedicated and code-slaves but if you are at it for long, sooner or later, you will suffer a burn out. Too much of anything is bad. Good companies always look for well-balanced individuals. Its good to indulge in different things. You never know what links to what. Sometimes when I am walking, a solution to some problem strikes me out of the blue. At this point I do a U-turn and jog back to my laptop. But its change which brings out the solution.

Every person needs fresh air. Whatever profession you may be!

To be a tester, be a developer (just for a bit)

Sometimes to know how green the grass is on your side just walk across the fence.

Lately at work I have been taking up the role of writing code which would actually ship. In other words I am not testing anyones code (except mine). And testing your own code is waaaaay too difficult than I thought it would be. But more on that in another post, some other day.

Here is a story. I checked in the first version of my program and awaited feedback. After waiting a day or so, and checking in another version, I walked up to the tester to check what was up (read: I tested my own code and found some bugs, did you find any more?). It seems she had apparently not looked into much of it. Not a problem.

Few more days pass by and no feedback yet. By then I had started doing what I love doing. Testing my own code and checking in newer versions. The good part of all this was after a few days I had really some robust code which I felt really confident about.

Yet, nothing beats another pair or eyes looking over your work. So by now I started pestering this tester to check my work. Finally I got comments from her which did put me in a state of partial shock. "Please change the way you have spelt CLick to Click." I had serious doubts in my mind. Is this a genuine comment after doing some solid testing or is this a "lets show I am on the job and keep him quiet for a while" kinda testing. Anyways, time went by and I didnt really hear from her. I got busy with some other modules and few days before shipping my favorite tester walks into my room and says she found a problem with my code. Now she did not pin-point the flaw but the code basically built a filter and and then did some analysis on the data which passed through this filter. As per the tester, there was certain data which should pass through the filtering mechanism but it apparently did not pass through mine. I wont get into the details right now, but it was difficult if not impossible to determine what that data was. All we knew is I had say 100 entries and she felt it should have been 101. All this few days before shipping.

Now changing my filter might potentially break the code somewhere else. Complete dilemma! And few seconds of anger for catching a bug so late in the cycle. Countless hours were spent stepping through my code. But somehow I couldnt figure out where I was going wrong.

Finally I walked up to the tester and asked her how she had come up with her numbers. It seems she had built a quick app which would do something similiar to what I was trying to achieve and had cross checked the results. Brilliant! May I have a look at that piece of code please? So I sat with her for a few hours and stepped through the code line by line. Finally we caught a bug in her code. Was I pissed off? No! I was relieved that my code worked fine.

What did I realize from this experience?

• A lot of heart beats are skipped when a bug is found very late in the dev cycle.
• At this stage when the dev. says "I refuse to change my code", he is pretty much justified. Of course this depends on the impact fixing this bug might cause across the application.
• As a tester if I do a bad job, it could thow others into a lot of trouble. A false negative is worst than a positive.
• More things which I will talk about later as the bell tolls.

But yes, being a developer for a wee bit has raised my bar of respect for developers. Specially those who do a good job.

And those who are wondering if I killed that tester, no I didnt. Everyone makes mistakes. Had she applied herself right from the start I would not have been grumpy. Unfortunately I did not see her do that. Every week I have to write a project update to my director. Also, everyone sends in a powerpoint slide to him listing what work they have completed in the week which passed by. My last bulleted point read something like "Did some testing. Found a bug in the code which tests my code." My director is a clever man. He understands the rest. :-D

In the Matrix

Update (4/18): This post does not seem to render well in blogger. You could find a well formatted version here.

If you belong to a testing organization its very difficult for you to do without test matrices. As a matter of fact you would be using test matrices semi-consciously in your day to day life. Managing money, time or girlfriends. You need matrices.

But on a more serious note, test matrices can be used for a variety of things. You could record a consistent set of tests using the test matrices. It can be used as proof of coverage of a certain criteria of tests, to serve as a checklist and refine/improve area in which a defect occurred.

Lets take an example to make things clearer.

Consider some new font sizes, size 75, 80, 85 and 90 which have been included as a standard in your Word application. You need to test whether they work well with your Word application across different platforms of Windows.

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80
Font size = 85
Font size = 90

Figure 1

What have we achieved here? We have set up two axis. The Y-axis contains font size and X-axis the OS. So we can put down the first step as determining the axis variables and putting them in such a form where you have minimum one test per row and one test per column. The test performed is recorded as the intersection of row and column. Once the test is performed, you could lodge the result as a pass or a fail.

You could use other notations in your test matrix too. P=Pass, or N=Note, or F=Fail etc. A test could pass but you would like to add a note to it. Very possible.

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75	PASS
Font size = 80		PASS	PASS
Font size = 85			PASS
Font size = 90		PASS

Figure 2

Some points to note from the above:

Highlighting gives us a better idea of test coverage.
Complete testing of a huge matrix is not feasible. The matrix would give us a good idea of test coverage for key areas.

You could apply your mind in different ways to eliminate or identify problems.

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80	FAIL
Font size = 85		PASS	PASS
Font size = 90

Figure 3

From Figure 3, we could conclude that there might be a bug with Windows 2000, there might be a bug with font size 80, both of the previous or there could be a bug when you try font size 80 on Windows 2000.

Lets test on the same axes as the failure and we would be able to make a much better guess to the problem. We perform two more tests.

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80	FAIL	PASS
Font size = 85	FAIL	PASS	PASS
Font size = 90

Figure 4

Ah! Now we know the problem is platform specific.

What if the matrix read:

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80	FAIL	PASS
Font size = 85	PASS	PASS	PASS
Font size = 90

Figure 5

Figure 5 will help us conclude that the problem is specific to Windows 2000 and font size 80.

One more matrix which could come about would be:

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80	FAIL	FAIL
Font size = 85	PASS	PASS	PASS
Font size = 90

Figure 6

Figure 6 shows a font specific bug.

So on, test matrices can be used for any pair of criteria's. Sometimes, a bug might be found which might not be listed under any criteria in the test matrix. The matrix can easily be expanded to include this criteria. Another advantage I see of test matrices might be the fact that if a defect is found after release, one can always go back to the matrix and look for areas not tested or areas which were tested and still had a bug (this could be poor testing).