Making the Software you use..... Better!

Testing. How do I begin?

2005-06-19T18:30:00.000-04:00

Anita asks, "I am trying to get into software testing..but I never took a course on testing..what would u recommend for a beginner???"

I could sum this up into three parts.

1. Read it.
2. Experience it.
3. Network with it.

Unfortunately testing is not something which might be taught at most schools. I was lucky enough to work as a tester under one of my professors in his team. Even there we were handed out certain notes on day one and asked to read up. There are many good books in the market. Some of the historical sites to refer to would be

Cem Kaner: http://www.kaner.com
James Bach: http://www.satisfice.com

Reading through their articles and publishings+recommendations would introduce one to the world of testing. If you have extra cash on you, attending a seminar/training sessions might be useful (I say might be since I have no on hand experience of attending one of these). I believe what could be taught could be very well learnt by yourself. Another major source (and a free one) is that of blogs. There are many testers blogging out there. One person I religiously follow is Michael Hunter (http://blogs.msdn.com/micahel). Just reading his blog gets you hooked onto testing. So yes, you need to start reading.

But all books aint gonna get you nowhere. Unless you practice it. If your day job does not support testing, make it your night job. If you write code, test it more sincerely. One practice I used to follow was trying to break my friends code for some project while he tries to break mine. Working in pairs could help. Try getting a partner. Not only does it make your code robust during submissions, but it also gets you better grades if you are in school :-)

Look up Mozilla (www.mozilla.org). Participate in the open source community. You could download nightly builds from Mozilla and try to break them. Want to know what a test case or a bug report would look like? You'll get those on Mozilla. Did you know Visual Studio 2005 Betas are being shipped free? Get one of those and try to probe into it. The more inquisitive you are about testing, the more questions you will have. And more the questions, more you will learn. Makes sense?

Sign up as Beta tester. Microsoft has a Beta testing program. Acronys has a Beta testing program. Try to look up some test automation tools. Dig into nunit and the likes.

Lastly, network around. Participate in discussions, write articles, offer suggestions. Not everything you do might turn up being right. But if you aint getting your fingers slapped once in a while, you aint pushing the boundaries far enough. If nothing else write a blog.

Start working on a pet project, test it and upload it to one of the communities. People will download it, run it, and test it for you. Vice-versa! Everytime they report a bug, you know where your testing+coding skills need to be improvised. Pick project ideas off Codeproject, Planetsourcecode or my professors site: http://www.ecs.syr.edu/faculty/fawcett/handouts/Webpages/Projects.htm
Buddy up with someone and build it, break it. Break it to build it. Try to automate your tests. Everytime you wonder how you could get a certain aspect of it done (ex. how to provide test hooks etc.), look up Google, ask people.

And of course, if you land up with a Software Testing job, it just makes life simple.

Reading up, implementing it and asking around is what helped me. There aint a formula to getting started. Just get started with one aspect of testing, it leads to others. How wide you spread your arms depends on your interest.

Quick advice before you venture into any coding. Write a quick Spec. Getting a clear spec is the first step towards being a good tester. Not only will it help you design, it will help you catch bugs before you start coding.

You can be a hero, but do you want to?

2005-05-24T20:10:00.000-04:00

Are you in one of those teams where testing starts late in the development cycle? Do your testers and developers work round the clock and spend late nights to finally get code to ship? Do you have those last minute moments of excitement and last minute heroisms?

A friend of mine, also a fellow tester was telling me about a project he had joint late in the development cycle which had a loose infrastructure for testing. How he and his team had to work late hours to ship the product out in time with minimum bugs. After pulling out such a feat successfully he felt like a hero, but also felt burnt out by the end of it.

This made me wonder. Do we really need heroes? Yes, we do. But its the Last-Minute heroes that we could do without.

Testing which starts late int he development cycle gives many bugs which might have been initailly small and simple to get entangled with some complicated ones and blow out of proportion. Now these bugs get difficult to fix. Now if you manage to identify them and fix them, invariably you will come out as a hero. But do you really need this kind of heroism?

Lets look at this from a different perspective. Lets talk about actual bugs. Cockroaches. Roaches to make it short. They can swarm up your place. I have seen this happen to a persons house. They were all over and finally he had to get the place bombed. But this could have been prevented. Getting a good vacuum cleaner, cleaning up the place regularly, using pest sprays when the first tiny roach made its appearance and maintaining hygenic conditions in and around his locality would never have seen them appear. This makes me draw a simple comparison between the vacuum-cleaners to people who bomb the house.

Vacuum cleaners are not the ideal thing to watch for entertainment. Its boring to vacuum your house everyday. Watching house-bombers in action is entertaining.
Vacuum cleaners come cheap. To get your house bombed would be expensive.
Vacuum-Cleaners are not the ultimate solution. Pests do get in even after maintaining a clean environment. But hopefully you wont need to bomb your house as often.
Using a vacuum cleaner requires you to do some amount of thought-processing. Where do you start? When do you start etc.
Estimating the preventive job vacuum cleaners do can be difficult. You cant count the number of insects killed.

Now lets talk about your software team. On your last project did your developers work in reactive mode? Did you keep sending them back at the last moment to get things up and running? Did you find bugs late in the development cycle? Did you have a loose infrastructure and finally manage to burn your team out?

But what if your team took cheap measures like the vacuum cleaners to do the work beforehand and get bugs fixed before they could get entangled into a big mass of code? Yes, you have these cheap tools at hand:

Automated testing tools. Yes, for the ones who are too lazy to get their own automation set up, there are cheap automation tools which would test code for glaringly suspicious constructs.
Spec reading. Yup, reading through your spec could help identify ambiguities or spec bugs.
Unit testing. Its getting famous!
Code reviews. Like I might have mentioned before, I might give up unit testing, but I wont give up code reviews.

All of the above, mainly the last one, look like the most boring things to do. Its definitely not on anyones list of exciting things to do. But code reviews is something all of my team goes through. The fact that it could very well find 70-80% of your bugs makes it high priority!

You see what I am getting to. Most of these techniques would help you avoid those nail biting finishes. They cost much less, are much less exciting and require lot of planning.

Who doesnt want to be a hero? We all do. But I would'nt want to be one where it could be avoided. At least not when it comes to testing software.

Passion Passion Passion!

2005-05-02T21:21:00.000-04:00

I have spoken about passion in a few of my posts. When you hire someone, look for passion. I happened to write about passion on my personal blog and thought I might just link it here for some reading.

The link is
http://heartcurry.blogspot.com/2005/05/passion-of-apoorva.html

Come what may, passion is the key!

On The Lighter Side

2005-04-26T21:52:00.000-04:00

Testers have a life too. Oh yeah, we pretty much do. Its usually the developer who spends sleepless nights. Dont believe me? I have someone to back me up.

I have been asked what else do I do besides testing stuff. First off, let me tell you that testing is more of a hobby right now. Its something I am passionate about but its not what I am into full-time. Where I presently work, I am not into full time testing. As a matter of fact we work on clinical trials. Work on creating programs which would generate certain statistics for a particular drug being developed. Yes, its very different from the normal software world. Why am I doing it? Just because its something new I am getting to learn. It gives me scope to draw an analogy of the pros and cons of the software/IT world compared to the pharmaceutical world. And it throws light on some different aspects of design and testing.

What else do I do normally? Well, most of the time you could find me behind my laptop. Either its some new thing I am trying out (you can spend a whole day trying to figure out inheritance even after you thought you knew it well - I'll see if I could post some examples later). Sometime I am looking up whats happening on Mozilla. Lately I am trying to look deeper into algorithms and data structures. Make my brain work that little bit more.

Then there is walking/jogging. It sort of depends on my mood. When there is a lot of thought process going on, I prefer long walks. It helps you clear out things in your head and get fresh in no time. Else I'd rather run on the treadmill. You'll probably find me at Bally's every alternate evening.

Cooking. yeah. I enjoy cooking. But I wont go more into that. But the day I open my own chain of cuisine, everyone shall be invited ;-) And then, there is music. I do play the key board but its been years since I touched one!

Why is this post a little off topic? Just to let you know its important to strike a balance. Companies love people who are dedicated and code-slaves but if you are at it for long, sooner or later, you will suffer a burn out. Too much of anything is bad. Good companies always look for well-balanced individuals. Its good to indulge in different things. You never know what links to what. Sometimes when I am walking, a solution to some problem strikes me out of the blue. At this point I do a U-turn and jog back to my laptop. But its change which brings out the solution.

Every person needs fresh air. Whatever profession you may be!

To be a tester, be a developer (just for a bit)

2005-04-19T20:49:00.000-04:00

Sometimes to know how green the grass is on your side just walk across the fence.

Lately at work I have been taking up the role of writing code which would actually ship. In other words I am not testing anyones code (except mine). And testing your own code is waaaaay too difficult than I thought it would be. But more on that in another post, some other day.

Here is a story. I checked in the first version of my program and awaited feedback. After waiting a day or so, and checking in another version, I walked up to the tester to check what was up (read: I tested my own code and found some bugs, did you find any more?). It seems she had apparently not looked into much of it. Not a problem.

Few more days pass by and no feedback yet. By then I had started doing what I love doing. Testing my own code and checking in newer versions. The good part of all this was after a few days I had really some robust code which I felt really confident about.

Yet, nothing beats another pair or eyes looking over your work. So by now I started pestering this tester to check my work. Finally I got comments from her which did put me in a state of partial shock. "Please change the way you have spelt CLick to Click." I had serious doubts in my mind. Is this a genuine comment after doing some solid testing or is this a "lets show I am on the job and keep him quiet for a while" kinda testing. Anyways, time went by and I didnt really hear from her. I got busy with some other modules and few days before shipping my favorite tester walks into my room and says she found a problem with my code. Now she did not pin-point the flaw but the code basically built a filter and and then did some analysis on the data which passed through this filter. As per the tester, there was certain data which should pass through the filtering mechanism but it apparently did not pass through mine. I wont get into the details right now, but it was difficult if not impossible to determine what that data was. All we knew is I had say 100 entries and she felt it should have been 101. All this few days before shipping.

Now changing my filter might potentially break the code somewhere else. Complete dilemma! And few seconds of anger for catching a bug so late in the cycle. Countless hours were spent stepping through my code. But somehow I couldnt figure out where I was going wrong.

Finally I walked up to the tester and asked her how she had come up with her numbers. It seems she had built a quick app which would do something similiar to what I was trying to achieve and had cross checked the results. Brilliant! May I have a look at that piece of code please? So I sat with her for a few hours and stepped through the code line by line. Finally we caught a bug in her code. Was I pissed off? No! I was relieved that my code worked fine.

What did I realize from this experience?

A lot of heart beats are skipped when a bug is found very late in the dev cycle.
At this stage when the dev. says "I refuse to change my code", he is pretty much justified. Of course this depends on the impact fixing this bug might cause across the application.
As a tester if I do a bad job, it could thow others into a lot of trouble. A false negative is worst than a positive.
More things which I will talk about later as the bell tolls.

But yes, being a developer for a wee bit has raised my bar of respect for developers. Specially those who do a good job.

And those who are wondering if I killed that tester, no I didnt. Everyone makes mistakes. Had she applied herself right from the start I would not have been grumpy. Unfortunately I did not see her do that. Every week I have to write a project update to my director. Also, everyone sends in a powerpoint slide to him listing what work they have completed in the week which passed by. My last bulleted point read something like "Did some testing. Found a bug in the code which tests my code." My director is a clever man. He understands the rest. :-D

In the Matrix

2005-04-16T00:57:00.000-04:00

Update (4/18): This post does not seem to render well in blogger. You could find a well formatted version here.

If you belong to a testing organization its very difficult for you to do without test matrices. As a matter of fact you would be using test matrices semi-consciously in your day to day life. Managing money, time or girlfriends. You need matrices.

But on a more serious note, test matrices can be used for a variety of things. You could record a consistent set of tests using the test matrices. It can be used as proof of coverage of a certain criteria of tests, to serve as a checklist and refine/improve area in which a defect occurred.

Lets take an example to make things clearer.

Consider some new font sizes, size 75, 80, 85 and 90 which have been included as a standard in your Word application. You need to test whether they work well with your Word application across different platforms of Windows.

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80
Font size = 85
Font size = 90

Figure 1

What have we achieved here? We have set up two axis. The Y-axis contains font size and X-axis the OS. So we can put down the first step as determining the axis variables and putting them in such a form where you have minimum one test per row and one test per column. The test performed is recorded as the intersection of row and column. Once the test is performed, you could lodge the result as a pass or a fail.

You could use other notations in your test matrix too. P=Pass, or N=Note, or F=Fail etc. A test could pass but you would like to add a note to it. Very possible.

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75	PASS
Font size = 80		PASS	PASS
Font size = 85			PASS
Font size = 90		PASS

Figure 2

Some points to note from the above:

Highlighting gives us a better idea of test coverage.
Complete testing of a huge matrix is not feasible. The matrix would give us a good idea of test coverage for key areas.

You could apply your mind in different ways to eliminate or identify problems.

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80	FAIL
Font size = 85		PASS	PASS
Font size = 90

Figure 3

From Figure 3, we could conclude that there might be a bug with Windows 2000, there might be a bug with font size 80, both of the previous or there could be a bug when you try font size 80 on Windows 2000.

Lets test on the same axes as the failure and we would be able to make a much better guess to the problem. We perform two more tests.

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80	FAIL	PASS
Font size = 85	FAIL	PASS	PASS
Font size = 90

Figure 4

Ah! Now we know the problem is platform specific.

What if the matrix read:

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80	FAIL	PASS
Font size = 85	PASS	PASS	PASS
Font size = 90

Figure 5

Figure 5 will help us conclude that the problem is specific to Windows 2000 and font size 80.

One more matrix which could come about would be:

	Windows 2000	Windows XP Pro	Windows XP SP2
Font size = 75
Font size = 80	FAIL	FAIL
Font size = 85	PASS	PASS	PASS
Font size = 90

Figure 6

Figure 6 shows a font specific bug.

So on, test matrices can be used for any pair of criteria's. Sometimes, a bug might be found which might not be listed under any criteria in the test matrix. The matrix can easily be expanded to include this criteria. Another advantage I see of test matrices might be the fact that if a defect is found after release, one can always go back to the matrix and look for areas not tested or areas which were tested and still had a bug (this could be poor testing).

U + I = UI

2005-03-26T00:17:00.000-05:00

I recently had to change my cell phone service provider. I would never have left Verizon (still claim they are the best) but somehow their service around my home sucks! So I mentally prepared myself for a phone. No digging deep into my pockets for a device which will play mp3's, allow me to IM, serve as a camera + video camera, have some colored tooth, internet browsing, windows media player etc. which also serves as a phone. Bottomline - control the geek in me..... and of course, spend minimum.

I ended up buying the Sony Ericsson Z-500a.

Now I am sitting home looking at this phone comparing it to my LG VX6000. OK, for its price and features, the phone aint bad at all. It lacks the the classy look of the LG. But the bad part is some of its design features.

Lets take this a little further. On your cell phone, you get into "Menu" and then hit "Contacts". Mostly you will have a list which will read as - "List Contacts", "New Number", "New E-mail", etc. Gee, now you realise that this aint what you were looking for. You were looking for the camera feature of this phone.

Next logical step: Hit Clear or CLR or "C" (as seen in most of the phones).

The user model of the clear button or anything which says "C" is you either go back to the previous screen or just clear out to your main screen. Oh well, thats how it has been with my LG and the phone we have back in India.

Back to the Z-500a. Hit the key "C" and the screen just stares at you. Hit it again and still no effect. Now look closely at this picture.

The Z-500a

There are 4 round buttons. The round button the green arrow points to is "C" or clear/cancel (whatever you wanna call it). The red arrow points to a button which had a u-turn shaped arrow on it. I figure this means "Back". So here is the key. Hitting "C" will not clear the screen or rollback. There is a special key to take care of this functionality. So whats the "Clear" key for? When you type in text and if you type texy instead of text and wanna clear the "y", you use clear.

Well, as a concept, its pretty clear. But for someone used to the "Clear" key taking care of everything, its frustrating. You hit the "C" key and wait. Then you hit it again and wait. You think something is wrong with the key. Finally you realise you have been hitting the wrong key. Is the UI bad? No. Its not. So that means the LG UI was bad? No, that was not bad either.

Eh? Am I actually going to learn anything from this post then?

Lets go back to what we are good at. Computers. Operating Systems. The Mac and Windows. Imagine if in the Mac, if you have to move a window, you can grab it by any edge and move it. In Windows, you need to move it using the title bar. For a Windows user working on the Mac, a task of resizing the window will result in moving the window. Very very frustrating. And you say the Mac is known for a poor UI? Of course not. Anything which comes out of the Apple store is known for its pretty face. The world claims the Mac to be pretty and Windows to be loaded with features. But thats a story for another time. The point here is ease of use.

Hmmmm... now we are getting somewhere. I might actually learn something out fo this.

In UI design, you have to see two aspects. User model and Program Model. You have to match the user model.

If your Program Behaves the Way users thought it would, then you have a well designed UI.

The mac OS works perfectly fine.... for Mac users.

The UI design has to be made keeping U and I in mind. You, I and the others. Usually ten of us.

Oh, by the way, the Z-500a does have a camera, video recorder, IM, web browsing, speaker phone, media, etc. And only worth $50. Not bad eh? Of course, I dont know if I am going to stick to it... that "C" key is very frustrating indeed.

I help, You Help. You Help, I Help!

2005-03-20T22:05:00.000-05:00

Till now I have been rambling about how it is important to have a good customer focus and how its really important to nail down the specs. Get something wrong and suddenly you are the target of a lot of customer ire. Ship a product with a glaring bug and find yourself losing a big chunk of your market share. But how much ever perfect one would try to be, a product will be shipped with bugs.

Today let me talk about how you, as a customer could help in making a better software.

Report bugs. Its simple.

Lets take Visual Studio as an example. A very well made software (yes, I know there are bugs, but looking at the complexity of the system - I think some bugs are expected). But here is how you can help. When you are working happily along and a terrible crash occurs, a much dreaded dialogue comes up which says something like -
“Microsoft Development Environment has encountered a problem and needs to close.”

And then a little farther down it says…

“Please tell Microsoft about this problem.

We have created an error report that you can send to help us improve Microsoft Development Environment. We will treat this report as confidential and anonymous.

To see what data this error report contains, click here.”

Then you see three buttons: Debug, Send Error Report, and Don’t Send.

Now this is where as a customer/user you could help. After all you are using the product. So why not make it better? Finally its for your benefit!

The above mentioned is known as Watson. When you hit the "Send Error Report" a minidump is sent to the server at Microsoft.

Why am I saying all this? Because I have seen people hit - "Dont Send". I know co-workers (where I work as of 20 March 2005) who spend a long chunk of their day on MS Word and have many crashes. When the Watson Dialogue pops up, they just say "Damn Word" and hit the dont send.

Unfortunately not many people know about Watson. I am sure if they knew what happens behind the scenes they would hit the "Send" button. Altho nothing can be as clear as - "Please tell Microsoft about this problem." But the user is anyways pissed off with Microsoft since what he was working on crashed (consider this user to be someone not highly adept in computers). Now when he finds another screen saying "report the problem" - irritated users says "NO". Dont ask me why - but I have seen this mentality.

Similiar with the Google pop-up blocker. Google suggests that if some pop-up manages to pass through the blocker, to report it. What page you found the pop-up on and properties. Now, how many of us would actually report that? Very few! How many would say "Damn this pop-up blocker!!!" whenever a pop-up appears? I see all hands go up.

This is a two way process. Software teams can only provide you a certain level of security. Testers try to advocate for every end user. But unfortunately thats not possible. The end user has to help the tester too. I can help you only if you can help yourself. The next time - do yourself a favor and hit that "Send Error report" button. After that you have the full right of saying - "Damn ".

The First Tiny Steps

2005-03-17T00:04:00.000-05:00

A reader left a comment on Michael Hunters blog asking how one could develope into a SDET (Software Design Engineer/Test) when his day job does not have much scope for automation. Michael provides him a simple answer which says to make it his night job. heh!

Reading this makes me relate to a few days back when we were preparing the final validation documents for submission. Unfortunately I cant go into details till our press release but I'll talk about something very similiar.

As a new employee 2 months ago I did notice that no one is really a big fan of automation. Some places do not need automation but some places could do with automation. Unfortunately I had not been able to convince my manager to automate a few things I felt should be automated. But that didnt stop me from thinking up other ways to get his attention towards automation.

When we are making our daily runs/builds

All programs lie in the PROGRAM folder.
All Logs (of corresponding programs) lie in the LOG folder
The outputs of the programs lie in the OUTPUT folder.

Fortunatly when we laid out the specs, we put forward a naming convention for the three. An example would be, for an output the corresponding programs and logs will be:
Program : ABCD 1.2.3.sas
Output : Table 1.2.3.doc
Log : ABCD 1.2.3.log

The final stages of our submissions process is something like this:
A folder is created for every set of program, output and log. This folder will be named the same as the output file, taking off the extension .doc. So our validation folder would carry another folder with the name Table 1.2.3 and this folder will contain the above three files along with other documentation.

Now the poor programmers (myself included) who wrote the programs are the ones responsible for making this folder with the above documents. So if you walk into a programmers office when he is creating this structure, this is generally what you find on his screen. Four windows of Windows Explorer open, having the program, log, output and validation folder. Then he searches through the list of programs and identifies the one he wants, copies it into the validation folder. Ditto for log and output. Then he realises he has to do that for another 80 more programs/logs/outputs... turns around, takes out his revolver and shoots you in frustration! After he is done with all the 80 folders someone comes and tells him they had got the numbering wrong and all the numbers need to be changed. He shoots himself.

So to stop myself and others from being killed, I came up with a small (but effective solution). Automate this process. Usually we work off SAS. Doing this through SAS might have taken a little more time (which no one had) but like Michael suggested, I made it my night job. A simple application in C# would not take more than a few minutes to bring about the desired automation.

The next morning I did a small demo before my manager. He was happy. 3 hours of work had come down to 3 seconds. The programmers were happy. Everyone was happy. But most important - automation had taken its first tiny step. Suddenly I had people come up to me and say - "Hey, can this be automated too?" - "If we can write a generic program for this, it can save us a lot of time" etc. I find myself much more busy, but much more satisfied! The programmers are smiling. They finally got to go home early! A very small 20 line program can change the way everyone looks at automation today.

So when your workplace does not really support automation - look around. Definitely there is something which could be automated. There always is. Even the smallest thing. Once it gets noticed, there is no looking back.

On a different note - most people in the pharmaceutical industry might not know much about C# but to my surprise not a single person at work had heard about it! My manager just walked in today and wondered about this other issue and if it could be solved better with C#. For everyone at work, C# = Magic! ;-)

The Microsoft Interview - Revolutions

2005-03-03T21:41:00.000-05:00

Interview 4:
I called up Ian after an hour and let him know I was down in the lobby. In a couple of minutes I found myself in his office. He said he was training someone on interviewing and wondered if it was OK with me to have him watch. No problemo! So now there were 3 of us. Ian started off with, "Lets write some code." Yay!! I was asked to write a program which was similiar to a mergesort. After writing it, I was asked to test it. While testing it, I was told to correct any errors I found. Writing code on a whiteboard is a whole new experience.

Advice 13: Practice writing code on paper or a whiteboard before your interview.

After finishing off with the above, I asked Ian how he would recommend testing some code like the one I had written. This led to a very interesting conversation where Ian walked me through a step by step process on how one would ideally test code and why certain things were done in certain ways etc. After this I was asked to test a program which would check for 2 rectangles overlapping. But I had to apply everything which Ian had taught me over the last 15 mins. Had I not paid good attention to what he was saying, I would have been in a fix! :-) I felt I did really well here.

We spoke general stuff about testing etc. and then Ian briefed me on how it would be to work at MS. If I was chosen for the group I was interviewing for, what my responsibilities would be, etc. By this time I started thinking I was almost hired which brings me to:

Advice 14: Read Advice 10 of Part 2.

Ian walked me to the lobby and asked me to wait for my next interviewer - Mike.

Interview 5:
Mike also happened to be a Test Manager and my last interview of the day. By this time I was close to being called a Zombie! I could have done with an energy bar which I had not carried. We started off initailly by talking about SDLC and the role of test. Then I was given a "simpler" puzzle and then a difficult one. I got the difficult one rather easily but took time on the simpler one. Heh! At this point Mike said he liked my thought process. Yes, yes, I was on Cloud No. 9. Then he got into the team player mode.

Mike: "I am a fellow tester who hates being disturbed or working as a team. As a matter of fact, the whole team might hate me, I dont care. You need information about a certain part of the project on which I am working. You might need my opinion, you might need my code. How will you approach me?"

My first thoughts were why hire such a person...? But I let those remain as thoughts. After talking generally about team relationships, my reply was a direct approach of walking to the person with a friendly hello, letting him know about the situation I was in and saying it would really help in taking this product dev/test process ahead if he could provide me with his invaluable guidance.

Mike: "Doesnt work for me. Leave me alone, get out of my room."

I assumed it was the fellow testers response and told him about various other approaches I might try. I got the same response to all of them.

Mike made me go through several such excercises and when I look back on them, I messed up in several of those. Some of my answers were actually "pathetic".

Finally Mike picked up his laptop, and asked me to test a certain feature on it. Apparently his desk was all filled with paperwork + desktops so I decided to keep the laptop on my lap (oh, now I know why they named it that!!). But Mike pulled up 2 cardboard boxes, put one on top of the other and said I should use them to place the laptop! After a while I was paying less attention to testing the feature and more attention to balancing the laptop on those boxes!

We finally spoke about trustworthy computing and some other areas where MS plans to grow. I was finally told the positions I was interviewing for. As Mike walked me to the lobby he asked if I planned to get a PhD. He said Building 19 was just across the street and I could walk it up. As I dragged a tired me across the street on a cold January evening, I knew deep inside that I had messed up the last interview. Only if Mike considered balancing his laptop a brilliant act, I might get this job.

Advice 15: Learn to balance laptops! Just kidding! But seriously, towards the end of the day, you will feel dead tired. But these interviews are probably the most important ones (what people call - as appropriate). Just think its now or never and give yourself one last push!

If you love writing code, solving puzzles etc. you wont be mentally drained out. As a matter of fact, even after 5 interviews I felt all pumped up. But my throat was killing me with all the chit-chatting I had done!

Amanda called me on the day she promised she would. "You came very close, it was a tough decision, but the groups you interviewed with have decided not to make you an offer." Oh well, it was a good excuse to go out drinking and drown your sorrows. A week later, Amanda calls while I am in the bus. Hmmm... maybe they made a mistake, maybe I actually got the job! "You came very close, it was a tough decision, but the groups you interviewed with have decided not to make you an offer." Errr... Amanda, you told me that last week. "oh I did, so sorry, I didnt note it down that I made the call. your file is still lying on my desk." Are you sure nothings changed in that file? We both laughed! Its okay... the bus stops right in front of the liquor store!

Advice 16: Know where the liquor store is.

The Microsoft Interview - Reloaded

2005-03-02T23:58:00.000-05:00

In this post I would be changing the names of those who interviewed me. Also, I would be changing the order in which I was interviewed (probably you would know why I did this as you read on)

The journey:

Flying Newark to Seattle is a good 6 hour journey. It doesn't really help if you have a 2 year old kid seated behind you who insists on crying for those 6 hours. Everytime I saw a passenger take out his laptop I wondered if he was a MS employee and if he was writing some C# code? Ear plugs made life a little easier and after a while I found myself in a cab headed to the Courtyard Marriott in Bellevue. After paying off the cabbie I was wished Good Luck for my interview. Eh? How did he know??!! I reached the hotel by about 11 PM and a note in my room read "Microsoft and the Courtyard welcomes you". I plopped into my bed and woke up halfway into the night, still wondering how the cab driver knew I was here to interview. Maybe I should have asked him what kind of questions to expect in the interview. He might have known!

I spent the day before the interview just chilling around reading some stuff and went off to sleep early. Once again I found myself awake halfway into the night. No cab driver nightmares this time tho!

As I walked out by 7 AM, I saw the breakfast lobby flooded with would be MS employees! Aha! No wonder the cabbie knew! The ride to Building 19 was a fairly short one. The cab driver happened to be from India, so we managed to speak a couple of Hindi words with each other. By this time I was really nervous. The initial form that was given, I filled it out in the wrong manner. There were some 10 others waiting with me and one by one a recruiter would come, call out their names and take them to their office. This other candidate who apparently seemed more nervous than me tried to get out his nervousness by talking with everyone around him. He actually walked up to me and asked "Why Microsoft? Which position at Microsoft and why that position?" Fortunately Amanda, my recruiter came out and called my name. I was rescued!

Initial HR:
Amanda had a sweet office and asked me if I would like some coffee, how my journey had been etc etc. Then she went into the regular introductory questions, and told me which groups I would be interviewing with. At this point I made it clear to her that I had mailed my university recruiter a week before my interview asking which groups I would be interviewing with. At that point I was told that it would be decided on that very day since positions keep opening and closing. Hence, I had no clue as to what these 2 groups are working on. She said she was aware of that situation and would inform the people I would interview with that I had not been informed beforehand.

Advice 7: Try to find out the groups you would be interviewing with beforehand. You can research on this group and with a little more information, you might come out as an "ideal match".

Amanda interviewed me for a while, asked about my various projects, which one I found more interesting etc. At one point I also asked her if I am shown the door after 2 or 3 interviews, does it mean my performance was rock bottom. She said she has no sure answer for this. She knows candidates who were interviewed by 3 people in a loop and got a job and others who had 7 interviews and were rejected. Saying this, she took me out to the shuttle and I was on my way to Building 17 for my first interview in the loop.

Interview 1:
Betty came across as a very friendly person and met me in the lobby. As we walked up to her cabin, she tried to put me at ease by asking questions like would I like a drink, do I like Seattle, the typical chit-chat. All I knew was its a blatant attempt to make me feel comfortable before she starts her attack! I put in the first question which got us going - "What role do you play around here?" I was a little shocked when she said she was a STE! I was expecting a Lead or a Manager to interview me. Upon asking her how long she had been here, she said about 12 years or so. By this time I was all confused. 12 years and STE? Fortunately she clarified that she came in as a contractor, became permanent, went on to be a Lead and then a Manager, missed getting her hands dirty with code, so after lot of talks with the management, she finally got back to STE.

Betty and I got along really well. We spoke a lot about testing and being my first interview I asked almost as many questions as I might ask in a year. We got down to testing where she asked me to generate test cases for reversing a sentence a.k.a: "I am a coder" -> "Coder a am I". She asked me if I had heard of this one before and I told her that I had actually written code for this couple of days back. I think I did pretty well at this and we spoke about the different scenarios, categorized tests etc. After I felt I was done, she asked me one little question and it made all my tests null and void! It was like having a house built and then realising that all the occupants are 12 feet in height after its built! Fortunately I had asked clarifying questions in the start, but had missed out on this one. Hence I felt it was ok! It was really one of those "I would never think of it" scenarios. But then, thats what a good tester would think of!

Advice 8: Ask clarifying questions before you start coding or testing. Think a lot before you say anything. Even if you know the solution beforehand - THINK!!

There was a point when I asked Betty few questions and she said these were the exact questions she was about to ask me. Finally she wanted me to write some code, but said we were short on time. So said she would simply ask me if I knew how to write code. Now it would look stupid if I took time to think on this one, so I immediately answered in the positive and she took me to the lobby for my next interviewer.

By this time my initial nerves were gone. I was enjoying what I was going through and my perception about the people working at MS had completely changed. This did not last for too long.

Interview 2:
Mark met me in the lobby and said that I was late. Apparently Betty had taken too long and now he did not have enough time. Ok... not really my fault. And Betty probably took 10 mins more than the alotted one hour. Did it really matter? Ok, maybe Mark had some meetings to attend. Anyways, when we got into his office his phone rang. This is his end of the conversation. "No, you know I dont do that." "No Betty, asking code is not my cup of tea. Bye." Here I was dying to write code, and the first 2 interviews were gonna go by without any coding!

Anyways, Mark got into a lot of management related questions. Actually we started talking about marketting after a while. On how I would sell the product to my customer. Is it better to call them at certain centres to give a demo or should you ship CD's with the product to people interested during the beta stage etc. etc. After a while I was wondering what position I was interviewing for. Personally I felt my answers were "OK" but there was one part which disturbed me. Everytime I start to talk, Mark looked at his watch. Everytime I take time to think of something, he says I have to be quick since I didnt have time. Initially I was confused and at the same time not thinking to my best. Frankly, I was not too comfortable with Mark's way of interviewing. But then I thought maybe it is part of the interview and he is trying to stress me out. I maintained my stance of "thinking before answering" and he repeatedly kept saying we are short of time. We did not talk about code, we did not talk about testing. We just argued about management aspects & marketting. A little about my projects and places where I felt I could have done better.

Advice 9: Even if you are applying for a design/test position, brush up on the basics of management/marketting/anything vaguely related. You could get general questions which will touch the basics of all aspects of software development.

So finally Mark said his time was up and if I had any questions I should ask while we walked to the next interviewers cabin.

Interview 3:
Mark took me to the office of Ahmed, introduced me, spoke to Ahmed for a couple of minutes and left. Ahmed came up to me, introduced himself and said I was early. Probably half an hour before schedule and wondered if I could wait in the lobby for a while. One person says I am late, the other person says I am early. By this time I was wondering if this was Microsoft? So I found myself waiting in the lobby, pretty sure Mark is typing a "No Hire" for me. Ahmed turned up after half an hour and said we should go grab some lunch. Great! He suggested we head for an Indian restaurant.

Advice 10: Dont second guess how your interview went.

Ahmed turned out to be a very friendly person and once again my opinion about the people working at Microsoft changed. He was a Test Lead and knew which restaurant makes good Indian food! He initially spoke a lot about himself and how his wife insists they should eat at the Indian restaurant at least once a week! While driving he asked me, "If I ask you to test something, what will you do?" Very open ended question. So I naturally asked him, "I will ask you what you want me to test." After that I spent almost one good hour figuring out how to test a calculator. I was asked to generate bug reports, test plans etc. A lot of thought went into developer/tester relationships. I think I did pretty well on this one. But still, no code! I was almost going to beg him to ask me to write some code when (miracle!!) he asked to write code (yay!). Generate all numbers which are multiples of 8, less than 80. I thought this is too simple, there has to be a catch in it somewhere... so I started thinking of all the possibilities where I could go wrong. Can you believe it, I asked clarifying questions in this scenario too!!!! Finally Ahmed assured me there was no trick in this, he just wanted to see my coding style.

Advice 11: Read the .net design guidelines.

We drove back to Ahmeds office where he gave me a name and a phone number and asked me to call that number after an hour. The person was a Test Manager and that would be my next interview. So I spent one hour after lunch sitting in the lobby. All the waiting was making me tired and drowsy. I could partly blame it on the Indian food.

Advice 12: Eat well, but eat light. Try carrying an energy bar with you. Eating small amounts between interviews is an excellent way to keep your energy level up. Also, carry a bottle of water with you. A lot of talking, standing and walking could make you dehydrated. Oh... and waiting too!

Wanna know if I made it further?

Part 3

The Microsoft Interview - Part 1

2005-03-01T21:46:00.000-05:00

I had interviewed with MS more than a year ago but did not blog about the experience since I thought its not ethical to do so. I was surprised to see JobsBlog ask for links to Microsoft interview experiences. So I thought, heck, if Gretchen is asking for it, why not do it!! And while I am at it, lemme throw in some free advice.

Note: I still dont find it ethical to give out questions that were asked. Altho I shall give out a few of them (which already exist if you Google for "Microsoft Interview Questions")

Lets start off right from square one.

Advice 1: Dont give up on applying. I changed the format/wording of my resume many a time and kept updating it with new projects/technologies I was working on. When you strike the right balance, you will get noticed.

- I had applied to MS sometime in Jan 2002 (for an internship) during our career fair. Not that I had stellar programming skills then, but MS was the only worthwhile company which came on campus. Then I applied again. Online. Then again I applied when they were visiting Syracuse in Oct 2002 or so. One fine day I got an e-mail from a MS recruiter where I had to answer some questions she had sent as an attachment. Didnt hear anything from her after that and I was asked to be patient. :-) Once again, they were in SU in Jan 2003 and once again I applied. Still no response. Mid 2003 I graduated and was training with a company in NJ (read: Working for free) when I got an e-mail asking if I was ready to interview with MS at the Syracuse campus.

- Oct 2003, I found myself interviewing for the SDET position. This was the first time I was ever interviewing in my life so I was all nerves. This was a half an hour interview at the SU campus. And I felt this interview was brilliant. I was interviewed by Navi Ahmed who was a Test Manager in the Windows CE group. Very friendly person who laughed at almost everything. He started off with regular questions as to why testing, why MS, spoke about one of my projects, discussed some OOD principles, asked me a coding question and then a logic question. Finally I was asked if I had any questions.

Advice 2: Be methodical in your approach. Talk out loud and clear. Even if you are stuck with a problem, talk about all the possible ways you are thinking of getting to the solution (even if they end up being not quite right). Also, if you are being tested in an area you are not comfortable with, make it clear. They would rather test you on your strengths. (Note: This does not mean you can show all your experience in C++ and when asked questions in C++, say you are not comfortable with it! But if you have limited experience in MFC and your interviewer goes into details of MFC, let him know you aint a pro!!)

- When I look back on my interview, few things stand out in my mind. My answers to certain parts of these questions were wrong. The logic part, I got it right, but it was not the best way to solve the question. I confused overloading with over-riding. Lots of disasters. What I noticed is I considered many options (including the right one) before I chose the wrong one. Where I rocked was the way I wrote code and the way I tested it. I was very methodical, clear and verbose in my explanations.

- Fortunately I asked Navi by when I would expect the results and if I dont hear from MS who should I get in touch with? I was given a timeline of 2-3 weeks and if I did not hear from MS, to get in touch with my college recruiter. The way the interview went, I came out thinking it was all over and I am definitely not flying to Redmond anytime soon.

Advice 3: Dont try to guess how well you fared.

- Almost a month went by and I had not heard of anything from MS. I thought they probably dont care about sending out rejections, but nevertheless, I mailed Lisa who was the college recruiter for SU. I got a quick reply from her saying she would check into the issue and let me know. After a week, I got a mail from Lisa that I was flying to Redmond to interview for a STE. There was some problem with the paperwork and the results had not reached her. She might have never noticed this had I not mailed her and asked.

Advice 4: Always follow up. Not only does it show your interest in the position, but it could actually get you a step closer to the job.

- STE??? I had applied for SDET (my second option was STE). But then, I had heard that MS fits you into the role they think you are best suited for. Probably they thought I was a better fit for STE rather than a SDET!

Advice 5: If you are inclined towards a particular position, dont compromise on it.

- Lisa's mail particularly said that someone from MS would get in touch with me regarding my travel schedule. About 10 days pass and I have no mail. So once again I got back to Lisa, asking if it usually takes this amount of time or is there some problem. Once again Lisa said she would look into it and the very next day I got a mail asking me about my travel plans. I was flying to Redmond and interviewing on 26th Jan 2004.

Advice 6: Once again, if anything you feel is wrong, get in touch with your assigned recruiter. They are highly friendly and helpful!

If you got so far, you probably wanna click the link below.

Part - II

Pairwise Testing...

2005-02-13T16:04:00.000-05:00

Have you ever felt that there are just too many cases to take care of? Too many options to try? That suppose you made a product like Word and Font size 8 works well with Times New Roman with red color, but what if you change the color to blue, or the size to 20 or the font to Arial.... or for that matter choose one of those 64 font types, a size anywhere between 0-72 and a color anywhere from deepest black to lightest white! These are thousands of combinations, how would you test your product for each of these values? Random testing is not an option. Neither is testing what you feel important.

We can save time and effort and find bugs efficiently by testing variables and values in combination.

Michael Hunter explains how to go about this scenario using pairwaise testing.

http://blogs.msdn.com/micahel/archive/2004/04/28/122702.aspx

Dave Fries has a post at:

http://weblogs.asp.net/davfries/archive/2004/06/17/158900.aspx

Dave also points out to the following link for a much detailed explanation:

http://www.developsense.com/testing/PairwiseTesting.html

I would suggest to start off with Dave's post, follow it up with Michael's and then go for the detailed one. Altho the last one (by Michael Bolton) clears up everything, go through Hunter's and Fries first and try to see if certain questions are raised in your mind. If yes, those will be cleared up when you read what Bolton writes. Try it as an exercise!

Manage This!

2005-02-11T00:20:00.000-05:00

Ever wondered what I am working on? The last couple of weeks have been the most hectic weeks ever experienced. A deadline which gets pre-poned by 3 weeks (which leaves only 2 weeks to get the work done) + shifting to a new apartment + heavy snow is an ultimate test of endurance. To date I have not really spoken much about what and where I work. The problem is, I really cant since the stuff I work on is supposed to be kept under wraps. Like many know, till less than a year ago I was working in C++, C# (more Microsoft related dotNet technologies). But slowly I have made a shift just out of pure interest and to broaden my horizons. Right now I work in a pharmaceutical company. And I am not working much on the dotNet framework or on anything related to C++/C#/ASP etc. Am working off SAS.

Let me explain a little. Since most of you reading this would be from the IT/ Software Dev/test background and not from the pharmaceutical side, it would be difficult to comprehend. But anyone who has heard of clinical trials will have a fair bit of idea of what is about to follow. Before a new drug comes out in the market, years are spent on R&D for this drug. Before its release in the market, it has to be approved by the FDA. To get the FDA's approval, clinical trials (tests) are conducted on a select group of subjects (the layman might call them patients). First the tests are conducted on animals, and after determining a certain level of safety, they are tested on humans (yes, each person who is being tested has to sign a consent form etc.). Finally these results are interpreted by applying certain statistical analysis and presented to the FDA. Here is an area where I come into the picture. All the data collected is firstly entered in the form of a table (called a SAS dataset, but you could compare it to any database table like ACCESS, SQL etc.). These tables are exact duplicates of the way data has been collected on paper for every subject. Lets take for example every subject will have demographic data, such as ID, sex, race, date of birth etc. So if there are 1000 subjects in the study, you have a table (or dataset) called DEMOG which has all the above demographic information for the 1000 subjects. So on, you have another 20-25 tables or more (depends from study to study) of the subject records which would include lab data, adverse events, vital signs, disposition, ECG, taken at different times as per the statistical analysis plan.

Okay, I went off track a little there. The point where I come in is data validation. First off I write programs to make sure the data entered is valid. Now, this might look easy, but trust me, its not. You have to go through the entire statistical analysis plan (call it the specs). Break it down into pieces, apply these pieces to the datasets and then put them together. If they give the same picture which the statistical plan does, half your job is done. Again, relating back to IT. Always go through the specs before you start coding or testing!! All of my team depends on the fact that the data which comes out to them is validated and correct. If they work on incorrect data, get their programs up and running and then we have to make changes to the datasets, you know how disastrous it could be! Hence I have to make sure that I (or for that matter anyone who works on the validation part) know the data & the Analysis Plan inside out. Its as good as knowing the specs inside out before you start coding/testing.

The next part of my job is to write programs which would perform statistical analysis on the data. Like lets say, finding the mean of the maximum dose levels administered to subjects who were above 55 years of age. Once again, this is a simple example. We can get much complicated with survival analysis, Chi-Square tests, regression models to name a few. Again there are different ways you could choose a certain population. But are you doing it the most efficient and correct way?

To round it up, after a lot of reviewing, these results are finally publised and sent to the FDA for approval. Thats the SLC for "shipping a drug" in a pharmaceutical industry. Looks simple? It is... but not as simple as you think it to be. And before you ask "Why?", I'll follow it up in Part-II. Till then... Apoorva Joshi highly appreciates your patience. ;-)

Job Opening

2005-01-31T18:15:00.000-05:00

I got this call (actually e-mail + call) and would like to pass it on. I dont know how good/bad the position is, but its primararily VB.NET, ASP.NET, JavaScript, SQL server kind of position at ConnectXcite. Description Follows:

"Seeking an exceptional individual for a .NET programming position, based out of Metrowest, Boston. The ideal candidate will be interested in our company’s growth and their own by extension. Qualities we desire include taking initiative, getting work done no matter what, passion for what you are doing and the ability to handle pressure. You should be able to step up as we grow.

We are only seeking someone who is aligned with our company’s goals -- meaning a dedication to developing, enhancing and customizing the product for the clients. You will be partly responsible for taking our company to the next level and will also be given an equity stake in the company.

Requirements:

Experienced in .NET technologies- VB.NET, ASP.NET, JavaScript, SQL server. (Oracle would be a good addition, but not required)

Technically, extremely smart. You will need to troubleshoot on your own. Need to be hands-on and able to code systems on your own as soon as hired.

Ability to work closely with clients, requirements gathering, and develop throughout the software install life cycle

2-4 years industry experience

Passion, positive attitude, client-focused, team player

Hard worker, reliable, results-oriented

Ability to think in terms of software reuse, libraries and methodologies (process)"

If you want to apply, just send me an e-mail with your resume at apoorvajoshi@gmail.com and I would pass it on to the person responsible.

Note: This is not for the company I work for or in no way do I know any other information about this company. I got the above information due to my contacts/network. I have no other information about this company whatsoever. And yes, they will sponsor H1B visas for the right candidate.

Hey Specsy

2005-01-18T22:58:00.000-05:00

Note: This Post is partly a repeat of my previous post, SDLC. Specs are the most important thing which a product team could come across, hence I have devoted this post to it.

Lets take the simple example where you plan to build a house. The whole process will require a lot of brainstorming before you lay the first brick. First, you look at the ideal location. Once you have that, you would look into the layout, internal arrangement of the house, how many rooms do you need, etc. You would probably hire an architect to do the planning. Next you would have a construction authority work on the type of material you would like to use, how many days you would like the house to be up and complete in etc. Once all these details are laid into place, the construction will finally begin. Before you begin the construction you would probably want to check every small thing possible, like the soil beneath (sometimes it could be the one in which the house might sink), weather conditions etc.

So be it with any product. Before you start coding, you check through the specs. Check through is a wrong choice of words. To put it mildly, you tear the specs apart. Think about every possibilty.

Lets go back to the house. Your architect lays out the entire plan and you start the construction. A few months down the line, the house is built and you move in. You love your new house but your master bedroom is on the eastern side of the house, which is right next to Interstate - I 95. Passing cars and trailors all night keep you awake. The guest bedroom on the other side of the house is perfectly soundproof and your visiting mother-in-law enjoys the perfect sleep! The guest bedroom is too small for you and your wife to move into... oh only if you had thought about this I-95 before!!

So is it with software. Having the specs does not mean the product is ready to code. Read the spec and start thinking outside the box.

Lets pick a small part of a spec as an example. In this case, lets suppose you are building a product for Company XYZ. XYZ has its small team of technical staff which pens down the specifications document and sends it to your company (usually in house projects have PM's or Dev Teams take up the spec writing respobsibilities). Now, if you belong to a company which believes in good development/testing practices, your first task will be to review the specs. So lets take the following as an example:

Specification version 1.0

1.Shall read from a database file at startup if one is defined. On process termination, it shall save any entered data to the opened file.

First off, company XYZ has provided you with specs, which puts you in a better position than many other people I know (yes, sometimes specs are not provided). If you take every spec and start coding, you have missed an important step. An ideal practice at this stage would be the dev. team starts a lot of designing (on paper) and prototyping. Some prototype code is a common practice. The test team starts working on determining test cases and reviewing the specs (the dev does this too, but somehow they always manage to finish the review much much earlier than test and are knocking to tests door to ask when they can start coding). I shall get into the mentality of these two teams later. For now, being part of the test team, your focus is the specs.

Lets look at no. 1. Shall read from a database file at startup if one is defined. Great! Works for me. Sounds simple! Here is the problem. What if one is not defined? What if the one defined does not exist? Is not accessible (locked?)? What will the program do then? The spec does not state that. Rather than making your own assumptions with what the product should do, go ahead to Company XYZ and ask them what they expect it to do. Provide some options:Option 1: "Should we say - File not found, enter another file"Option 2: We could say, "File did not exist, taking demo file and displaying results to show functionality of product"

You have an ambigious spec clarified. A potential flaw has been averted! Imagine if you had taken your own decision, displayed Option 1 and during qualification (demo) XYZ says they wanted Option 2. The change in this case might take a few hours but imagine something huge for critical systems? Also imagine if this chage you make breaks something somewhere else? Wouldn't you rather spend a few extra weeks on the specs and get them clarified than have clarifications after the product has been developed?

Lets go back to the house. You finally cant bear the vehicle noise on I-95 and you break a few walls, make the guest room bigger and move in there. In this process you are set back by a few thousand dollars and possibilty of maing your house structure weak.

So will it be with any product. A bug fixed towards the end of the product development cycle not only costs more to fix, but could break the software in some other area, making the whole product weak in its structure.

So irrespective of you being a test or a dev or a PM, just forget what you are, think you are a team, and work with each other to produce a brilliant spec. You will be doing everyone around you and yourself a huge favor and missing out a few sleepless nights.

My OS... on the Web!

2005-01-18T22:52:00.000-05:00

A thought which crossed my mind (ok, I admit it was triggered off by something I read) but somewhere in the future, gone are the days of your desktop having an Operating System. How about having an OS on the web? All I do is push in an internet cable into my PC and it connects to my web-OS. Anywhere I go, I have access to my OS. I can log onto my PC from India, I can log onto my PC from Alaska. It doesnt matter if I log in on a P-4 or a P-1. My speed wont be affected since my OS runs off a server and not off my desktop. Windows, are you in trouble? Or do I see a Longhorn - Web come up few years down the line? If not... Google, are you listening?? For that matter, any other startup, are you listening? I have a brilliant idea...!!

I think its going to happen (who had thought of 1 GB e-mail space few years ago?). Its just a matter of time!

Talking about Google, here is an excellent move by them to make comment spams redundant!

SDLC

2005-01-16T22:42:00.000-05:00

Pete had previously asked, "You had mentioned that this blog would also be for those new to the field of software testing. I would fit in the "New" category which also gives me the excuse of asking basic questions. What is the Software Development Cycle and how does Testing fit into this Development?"

Good question. Like Pete many might not be aware of what exactly does System Development Life Cycle (SDLC) exactly mean in the software world.

Lets take the simple example where you plan to build a house. The whole process will require a whole lot of brainstorming before you lay the first brick. First, you look at the ideal location. Once you have that, you would look into the layout, internal arrangement of the house, how many rooms do you need, etc. You would probably hire an architect to do the planning. Next you would have a construction authority work on the type of material you would like to use, how many days you would like the house to be up and complete in etc. Once all these details are laid into place, the construction will finally begin. So everything goes by a particular method or process.

Lets get to the definition of SDLC provided at http://www.webopedia.com/TERM/S/SDLC.html It states:

(quote)

SDLC is the process of developing information systems through investigation, analysis, design, implementation and maintenance. SDLC is also known as information systems development or application development. SDLC is a systems approach to problem solving and is made up of several phases, each comprised of multiple steps:

The software concept - identifies and defines a need for the new system
A requirements analysis - analyzes the information needs of the end users
The architectural design - creates a blueprint for the design with the necessary specifications for the hardware, software, people and data resources
Coding and debugging - creates and programs the final system
System testing - evaluates the system's actual functionality in relation to expected or intended functionality.

(unquote)

Lets take a case where a company decides to develope a new product. First, they need to identify the need for this product. Why would it be required in the market. Next, how the end user would use this product. What would they want out of this product and what the market lacks right now which this product will provide.

Once these have been identified, we would probably have the Pragram Manager write down the specifications of the product. This is a document which has details of every small requirement that the product should meet. Right from the kind of platform it should work on to how its final output should look.

Once the specifications are up, the Development team starts putting them down into code and test team starts testing the code written by the development team.

Did anyone spot an ambiguity? Anyone?

Lets go back to the house. Your architect lays out the entire plan and you start the construction. A few months down the line, the house is built and you move in. You love your new house but your master bedroom is on the eastern side of the house, which is right next to Interstate - I 95. Passing cars and trailors all night keep you awake. The guest bedroom on the other side of the house is perfectly soundproof and your visiting mother-in-law enjoys the perfect sleep! The guest bedroom is too small for you and your wife to move into... Oh!! Only if you had thought about this I-95 before!!

So is it with software. Having the specs does not mean the product is ready to code. Read the spec and start thinking outside the box. First point of the spec says "Client shall take as input a file using the file dialogue open window" (this is a lame way of writing a spec, I am just trying to provide an on the spot example). Anyone try to think what will happen if I choose a file and before I hit open, someone from another machine with access to the server deletes that file? How should the system behave? What if I enter a filename which is not present? How will the system behave? Rip that spec apart.

The same goes for software, a bug in the spec is much cheaper to correct rather than a bug in the product once the code is up and running.

A good tester would not only find bugs in the product, but would find bugs in the spec. This is where testers come in. I would also like to mention here that developers also go through specs and try to identify bugs, but testers are more concentrated on developing test cases for the spec, developers for design prototypes for the specs.

Back to SDLC. And once a lot of reviews and discussions and prototyping is done, the dev team starts coding and test team continues testing till the final product is complete.

During this entire process the management team oversees the progress, makes decisions about new hires, deadlines, addition/deletion of new features etc.

I think I am going to add a new post of why its important to analyze specs before we start coding + some other stuff.

Some Questions, Answered...

2004-12-30T17:31:00.000-05:00

I had mentioned it before that I follow Michael Hunters blog pretty religiously. A while back I sent him a few questions about some issues which needed a higher opinion. He was pretty quick in getting back with a reply, which happened to be in form of a blog post.

Definitely worth a read. You could find the post at:

http://blogs.msdn.com/micahel/archive/2004/12/29/343778.aspx

Updated 8th Jan 2005, another post to a question I had asked Michael.

http://blogs.msdn.com/micahel/archive/2005/01/05/346964.aspx

Thanks Mike!!

Also, check out his blog titles. Always found them innovative.

Hats off to Dr. Fawcett

2004-12-25T18:49:00.000-05:00

Its the season to make merry, enjoy, have fun, overeat... u name it! With this I thought let me veer off to a different topic and talk about one person who is been instrumental in introducing me to the area of Software Dev. & Testing. Altho' I used to be a lot into programming during my undergrad years in Bombay and had an internship which was programming oriented I got a real taste of what things are at Syracuse University under the guidance of Dr. Jim Fawcett. As a matter of fact I just got a mail from him today and had a few comments about my blog. And it struck me that I should have a post acknowledging the person who has a big hand in what I am today. I took a lot many courses under him and also had my Masters Project under his guidance.

Dr. Fawcett provides as many as 7 courses in Software Development at Syracuse University. One of the courses which I cherish is Software Design Studio (CSE 784). The Software Foundry Project we worked on will live in my heart forever. It was this project where I worked as a pure "Software Test Engineer" and since then there is been no looking back.

You could visit Dr. Fawcett's web page at:

http://www.ecs.syr.edu/faculty/fawcett/handouts/webpages/fawcetthome.htm

And the first pic that you see, thats me sitting right in front of the PC next to Dr. Fawcett. It was during our Software Foundry Qualification Test. You could also check out the Foundry web page at:

http://www.ecs.syr.edu/faculty/fawcett/handouts/webpages/Foundry.htm

One more thing which the courses indirectly gear you up for is how to go on without a few days of sleep. Oh well, its not that bad, but for someone like me who has various activities besides coursework.... I enjoyed those sleepless nights.

Dr. Fawcetts courses not only thought me C++, C# etc. but a good methodical approach to any given language. Recently at work, where I initially worked off Microsoft based stuff and slowly learnt up other languages (eg. SAS), the learning process was quick. As a matter of fact, my new job is more based off SAS and little off .net. The courses taught me an approach to design in a way I could have only dreamt of. And more so, tight deadlines, learning new things on the fly and incorporating them in your projects have given me a good sense of how to approach anything new.

Anyone who considers Syracuse for CE/CS has to pick up a few courses under Dr. Fawcett, else you have not had a taste of Syracuse. Hats off to Dr. Fawcett.

Now, a little bit of a pop-quiz for you likely testers. Directing you to Michael Howards blog (who I believe is the author of Writing Secure Code and also works at Microsoft). The blog is on the evils of strncat and strncpy. Try to answer it without looking at the answers page. Just to let you know, I didnt get some of them!
http://blogs.msdn.com/michael_howard/archive/2004/11/02/251296.aspx

Also worth checking out is Sara Fords blog, with its entry on test case code and identifying false negatives
http://blogs.msdn.com/saraford/archive/2004/12/14/307153.aspx

You guys have a Merry X-mas and dont over eat! Time for me to see Heat taking on the Lakers!! I hope Shaq and Kobe put up a good face off!

Catch 'Em Young!

2004-12-17T01:34:00.000-05:00

At Syracuse, I was Teaching Assistant to Prof. Plumley. It was an engineering course to High School Students and we did a wide variety of things, from building rockets to physics, civil engineering, robotics, windfarm reports and lots of other crazy stuff. On the whole it was a fun course which the students and teachers, both enjoyed. I have continued this teaching habit from time to time. Myself and Prof. Plumley would work on most things together but the C/C++ programming part (lecturing the students) was left completely to me! Probably the most difficult time these students had during the course was learning C/C++ (and no, its not because I was solely incharge).

The course was an introduction to Engineering, so we touched most aspects of engineering and science. It gave the students a good idea which discipline of engineering they would like to get into. But finally I had students I could fit into 3 catagories. Those who end up in love with programming (and some actually read this blog), those who know programming is not their cup of tea, and those who are still confused.

An interesting incident I remember when I was intoducing the course to them. We were talking about algorithms, and I went about giving them a simple example. Their assignment for next week was to get me an algorithm which could add any two numbers between 0 and 99. I got the regular answers from all of them, except one. The one which stood out went about as:

Step 1: Pick a Calculator
Step 2: Enter a number between 0 to 99
Step 3: Press the "+" key
...........
...........
etc. etc.

I found this to be highly amazing. And this is where a good tester comes into play. He needs to fill in the boots of the regular users (most of the students above) and of the not so regular user (the one student who thought of a different manner in which the algorithm could be written). And those who are wondering, I did not penalise him for his approach, but just called him up to my office, told him I loved his approach and asked him if he could tell me another way to do it. He did have an answer to it and he got his high grades!

As we move ahead in the course, I ask the students to program the addition of 2 numbers using C/C++ and once they are done, students exchange programs and they are asked to test the others program. Finding bugs gets them bonus marks. And the student whose program is bugged, if he can tell me how to resolve it, he doesnt lose out on too many marks. Everyone found this exercise very entertaining and when I presented some test cases no one had thought about, it had them thinking.
All I did was give them some scent of the prey and those who were really hungry would hunt it down.

Testing is not only about sitting at the PC and banging away on code, trying to break it down. Its about spreading awareness on how to make the world a more secure place. Its not about improving only your testing skills but about improving the skills of your co-workers or for that of your own organization. Its about setting (and living) high standards that other people will look up to. And try to emulate.

Are you there yet? I feel I have just started!!

On a lighter note, a sweet answer for "Why do we test our code?" by one of the students was, "To improve our grade".

Put these on your car

2004-12-14T19:53:00.000-05:00

Harry Robinson is Test Architect for Microsoft's Engineering Excellence Group. Here is an interesting article he wrote up:

http://lnk.nu/stickyminds.com/nw.asp

Check it out!!

And sorry for not posting anything concrete in a while. Am working on a few articles and should get them up by the end of the year (hey, thats not too far!!)

Google v/s MSN

2004-12-03T14:16:00.000-05:00

As of now, Google is the mother of all searches. Smart hires and difficult interviews to get through into Google (at least the US locations).

Microsoft is warming up on the search front and altho BillG admits that Google is miles ahead, promises Microsoft will produce a much better search engine few years down the line (maybe 5?). The problem is just like how almost all people expect Windows on their PC, most people expect Google in their search. Just like its impossible (almost) to change laymen from Windows to Unix/Linux, its going to be difficult to change people from Google to MSN Search. Create a better engine, but how do you change the mindsets of the people? Marketting? Maybe... but nothing beats good quality.

Anyways, just a funny thing I found out... I was just trying MSN Search and comparing it to Google Search. So I type up "Apoorva Joshi" (yeah yeah, I am in love with myself) in both.

Google gives me 37 hits. But there is no mention of this blog or my personal blog.

MSN gives me 6 hits, and it does count in http://heartcurry.blogspot.com (my personal blog).

Interesting! I never expected my blog to show up really. With Google it was out of question, since its spiders would find no links. But it makes me wonder, what is MSN using that it missed out on 32 links which Google caught up, but got through into this one link which Google did not. I am no search engine expert so sorry if this question sounds like an average dumbass one to you.

Irony of things, I have posted on many MSDN blogs about stuff and it shows up in Google, but not on MSN. Whereas blogger is a Google based software and it shows up on MSN and not Google. How about searching your own house before you look into your neighbours buddy?

Innovative Google

2004-11-25T13:11:00.000-05:00

One of the things which makes Google stand out is the manner they come out with innovative stuff which is simple and identifies with every user. Apart from the great search engine and other applications they provide, there is a sense of belonging they create. Those who have a Gmail account would have seen this message when they get to the gmail site. For those who dont, here it is copy pasted from the site (I hope Gmail team wont mind that). I give full credit to them:

(quote)

A Gobble approach to email.

In 1621, a few hundred Pilgrims and Native Americans sat down to celebrate a bountiful harvest. The feast lasted three days, and included fowl, venison, fish, berries, watercress, lobster, dried fruit, clams, and plums. There was no pumpkin pie, however. There was also an alarming lack of user-friendly webmail services.

Now, 383 years later, it's once again time to celebrate what has come to be known as Thanksgiving—a time to gather with family and friends and give thanks for all that we have. We have many things to be thankful for. But mostly, we are thankful for you—our users—who remind us of why we work so hard all year and why we love what we do. That's better than all the dried fruit and clams in the world.

Happy Thanksgiving! Thank you for making our approach to email yours.

Gobble gobble,

The Gmail Team

(unquote)

Not to mention the innovative designs of the pie and turkey which they have incorporated in their Google & Gmail signs.

Just wondering, why dont other big software giants think of something like that with their products?

Career Killers To Avoid - on MSN

2004-11-19T09:52:00.000-05:00

Just came upon this article in MSN (careerbuilder). The author says, "I conducted a national survey of CEOs and top executives career paths and hiring decisions. From the results, I identified traits, ideals and characteristics that successful leaders seek in developing their workforce."

Here is just the bold points of their thoughts, from higher to lower priority:

Lack of Results.
Not Working at a Job you are Passionate about.
Not having big enough goals.
Thinking that money is everything.
Having a Bad Attitude.
Gossiping and playing Office Politics.

A month back, some of my friends and me had a strong discussion about the second point, that passion is a must. Passion plays a big part in who you are and what you become into. Huge corporations have lecturers coming in to tell you about passion and its importance. The best part about passion is it cannot be faked. More on passion at some later date.

I didnt resolve all the bugs. Am I a bad tester?

2004-11-08T14:47:00.000-05:00

Those who work in the industry would probably agree with what I type and those new to the field of testing might see this as an eye-opener. In my very first interview for a Software Test position, I was asked when would I say a product is good to ship? My quick answer was, "When there are no bugs in the product". My interviewer laughed and it struck me that I was wrong. So I went ahead to say, "Usually its not possible, but I would really like to ship a bug free product". All he said was "Good Luck". Needless to say, after a week the HR called me up to say they were sorry but I had missed out on the job.

When I think back on this, I laugh. Yes, who would'nt love to ship a bug free product. But I can almost gaurantee you that if someone tells you he has shipped bug free products all his life, either he is living under some weird delusion or he is taking you for a ride.

Products (and I mean almost all products I have known till now) are shipped with bugs. But these bugs are categorized into two types:

Bugs which are not known
Bugs which are known

For the amateur tester: Yes, products are shipped with known bugs too.

So if I know that my product has bugs, why the hell would I ship it?

Its something we all crib about and the word is "deadlines". Everyone has to meet a deadline and any good tester would know that testing never ends. So you have to put your foot down at one stage and say the product is ready to be shipped. And for me, this is where a good tester shines. Here is where you could prove your mettle. This is where you make the trade-off. You know there are bugs in the system, but how crucial are these bugs?

This is why I follow a mantra, which says "Prioritize". Every time you note a bug, give it a priority. Make sure the high priority ones are resolved. There is no way I would sign over the dotted line if these are not resolved. Try anything in the book to make sure high priority bugs are resolved before you ship the product. If the dev team does not agree to comply, camp outside your Test Managers office, sing a song in a lousy voice (I know one chap who tried this!!) and agree to stop only if the dev. team agrees to resolve your bugs.

Here is an example (I love examples). Lets take a product we use each day. Lets say, MS Word. Imagine you are a tester for some particular feature of MS Word. You realise that when you hit Ctrl+S your cursor moves back by 4 spaces, but your document is saved. So its surely a bug. Now each time you choose "Arial" font your application throws an exception and hangs. Again a very valid bug. But the second bug is definitely high priority. In a pressure situation I would ship the product with the cursor not staying where it should, but I would never ship the product as mentioned in the second scenario. My priority is to get the second issue resolved and I might ship the product with the first issue not resolved.

That took care of bugs which are known, but what about bugs which are not known? Usually its entered into your bug tracking system after user feedback. Like I said, its sometimes only the customer who identifies some amazing bugs we never thought of. Here again, you need to prioritize and hope to get it resolved in the next release. But if you have a major issue, get ready for some sleepless nights.

So if you didnt resolve all the bugs, you aint a bad tester, as long as you got the ones which really matter resolved. And this does not mean you take it easy. Always look for ways to convince dev. to resolve the low priority ones if possible. Keep slipping in the easy ones with the tough ones too! Keep that testing going and keep knocking your dev's. door till you make them cry (such an act tho', could be injurious to your health at a later stage)!

Bring In the Customer: Part - II

2004-10-23T17:14:00.000-04:00

....Continued from an earlier post. Bring in the customer early? But how early? Show the specs to him? Ask his opinion even before you start coding?

As I had mentioned the train scenario.... being the end user, would I ever have guessed that the seating was not proper by just looking at the design? No, I would not have! The same goes with any software. The end user just uses your product. When he comes across a problem, its reported in form of a bug and fixed in the next release (hopefully!). This end user would not be able to find flaws in the design stage. That part is left to us experienced testers. But there are few bugs which come out only when the product goes in the market. There is definitely some combination which we have not tested the product on. (An example, gmail worked fine on IE 6. Back in Mumbai I got onto my P-I which still runs IE 3.0 and gmail did not work). There are millions of these combinations which sometimes even if known are impossible to test due to time constraints.

Enter Beta Testing. A good idea is to release the product one stage before its ready to ship. Fine, in such a case, you cant make major changes. If someone says the train carriage size needs to be bigger, its not possible. But if someone says the seating needs to be rearranged, that could be possible.

Hence leave the initial part to the experienced clan. But get in the customer when appropriate. You would be surprised the amount of bugs which come in from Beta testers which make us wonder "How did I miss that??". Which is why a good tester has excellent customer focus.

Testing Never Ends...

2004-10-21T04:06:00.000-04:00

This is an interesting story I found while reading Micahels blog. Micahel is a test lead at Microsoft and writes some neat stuff on testing. For all testing afficianados, his blog is a must read. I treat it like a bible! A part of his blog read.

"A speaker demonstrating how testing is never complete told us that he was from Israel, where there are lots of places a plane can fly below sea level. Five months before the F-16 (the first completely fly-by-wire fighter, he told us) entered production, the F-16 team did some flight tests in Israel. Anytime the pilot flew below sea level, the plane turned itself upside down and flew along just fine! Five months before production!"

Testing never ends....

Interviewing Tips

2004-10-15T02:08:00.000-04:00

This one comes in reply to a S.O.S. A friend of mine mentioned about getting a call from Microsoft for their initial on campus screening through college recruiting. He also wondered how he could prepare for it. Here is what I have to say:

Look up their website. Look up their positions (I think they have 4). Go through the details and figure out which one you would identify with. Also try to identify with a core Microsoft technology and express your desire to work in that group (this is not a must, but would show you have done some homework!)
Once you have decided what you are likely to go in for, google around a bit for those positions. Many interview questions are available on google itself.
If you have the time, look up Zoe and Gretchens blog on http://blogs.msdn.com/jobsblog They have some good inside information on the recruiting process.
Irrespective of what position you apply for, make sure you are well versed in regular string manipulation, data structures and algorithms and the regular HR questions. I suggest Algorithms and Data Structures and Pointers in C written by Yashwant Kanitkar for the same.
Be prepared for puzzles. I think http://www.techinterview.org/ is a good place to start off with.
Most important: GO THROUGH YOUR RESUME!! If you cant answer something on your resume, dont show it there. "I had worked on this a while back hence I dont really recollect it" is a strict NO-NO!
Dont be scared to say "I dont know." Suppose you are asked questions on Operating Systems and its not what you want to go into, dont be scared to say "This is not what I would like to go into. Altho I know OS is an important part of MS etc. etc. but I would like to work in groups such as encarta or messenger. OS does not really interest me right now". These guys are more interested in your strengths and not your weaknesses. Of course, dont be too rigid.
Do a self evaluation. If you have taken specialised courses in Software Design and none in testing (and you know nothing about testing), you probably are more suited for SDE. Find where your passion lies and interview for that position.
Think! Think ten times before you code. Ask millions of questions. Test your code. I am sure the interviewer will ask you to test it anyways.
At the end of the interview, ask some genuine questions and not for the sake of asking questions!
More importantly, be yourself. Be calm. Be relaxed. Be methodical... and smile! Show the person across the table that you are enjoying the experience!
There is a remote possibilty that the person interviewing you is HR and not tech. Such times, tech. questions would be rare. Concentrate on teamwork, how you contibuted "differently" to your project and few examples of leadership qualities.

Good Luck!!

Bring In The Customer: Part-I

2004-10-14T15:54:00.000-04:00

As a software tester many a time I have seen myself get into the mode of a critic. This "critic nature" varies as per the phase of a project I belong to. Specially if I am in the Spec review stage this nature is at its best and never restricted to the software I am testing.

Yesterday as I travelled from Churchgate to Andheri (reminder: I am in Mumbai, India at this moment) via train during rush hour, this critic mode was at its best. An over packed train, people hanging out by the door, 4 people sitting in a 3 seat bench... the regular rush hour scene.

Here is what I wondered, there is a large amount of people who stand in the trains. The ones who sit are pretty uncomfortable since 3 seaters are occupied by 4. This in turn reduces the amount of space available to stand. On the whole, those standing and those sitting are uncomfortable.

Why not make the benches a little smaller, such that they fit exactly 3 and there is no place for the fourth? This way those who sit are comfortable and it opens up more space to stand. The way the seats are designed and laid out give minimum area for those who stand. A good design to implement would be SOME of the subway designs in New York City. All the seats are lined up against the walls (periphery) and the center space is for people to stand. I could keep going on with other suggestions I had...

Now where does Software Testing come into all this? This train journey is an ideal example of me being a customer/end user and whoever designed the train cars being the development group. Some issues which cannot be identified by the design/test group can be identified by the customer.... since the customer is the final user of this product. Those who designed the train cars probably had the idea of providing 3 people comfortable space to sit. A rush hour scene, where 4 people would sit on those benches, more people would be standing etc. might not have been imagined. But who experiences this scenario? The customer! Who gives feedback on these usability features? The customer! The customer is the most important aspect for any tester (or developer). More so for the tester since it is every testers role to advocate for the millions of users who would use the product.

All this makes me ask.... when should you bring in a customer into product development? We have Beta users. But what would happen if you bring in customers at the very initial stage. Advisable? I'll cover that in my next post.

An Introduction...

2004-10-13T16:30:00.000-04:00

Hello! This is Apoorva Joshi and this blog is for all hopeful Software Test Engineers as well as experienced ones. More importantly this blog welcomes anyone who is not of the software world... its always a good challenge to explain a non-tech person on how things work in the tech world. Every tried explaining your Grandmom how a C++ compiler works?

Who am I?

Call me an amateur tester. I test for fun. Sometimes my job deals with testing, which again equates to fun. I hail from the city of Bombay (renamed Mumbai), India. Currently I live in the United States of America. I am bad at self-publicity hence anything more you would just have to ask.

What this blog would cover?

Anything and everything which is related to Software Testing. The mindset of a Software Test Engineer, few experiences, interviewing tips and any other suggestions you would have. Most probably I would pick some Microsoft technologies as an example from time to time. Yes, I am a fan of Microsoft (no I dont work for them) and the way they take testing seriously (of late). More so this blog is aimed at creating an awareness of testing. Why should one take testing seriously. Its a small start to making this software world a more secure place.