February 13, 2005

Pairwise Testing...

Have you ever felt that there are just too many cases to take care of? Too many options to try? That suppose you made a product like Word and Font size 8 works well with Times New Roman with red color, but what if you change the color to blue, or the size to 20 or the font to Arial.... or for that matter choose one of those 64 font types, a size anywhere between 0-72 and a color anywhere from deepest black to lightest white! These are thousands of combinations, how would you test your product for each of these values? Random testing is not an option. Neither is testing what you feel important.

We can save time and effort and find bugs efficiently by testing variables and values in combination.

Michael Hunter explains how to go about this scenario using pairwaise testing.

Dave Fries has a post at:

Dave also points out to the following link for a much detailed explanation:

I would suggest to start off with Dave's post, follow it up with Michael's and then go for the detailed one. Altho the last one (by Michael Bolton) clears up everything, go through Hunter's and Fries first and try to see if certain questions are raised in your mind. If yes, those will be cleared up when you read what Bolton writes. Try it as an exercise!

February 11, 2005

Manage This!

Ever wondered what I am working on? The last couple of weeks have been the most hectic weeks ever experienced. A deadline which gets pre-poned by 3 weeks (which leaves only 2 weeks to get the work done) + shifting to a new apartment + heavy snow is an ultimate test of endurance. To date I have not really spoken much about what and where I work. The problem is, I really cant since the stuff I work on is supposed to be kept under wraps. Like many know, till less than a year ago I was working in C++, C# (more Microsoft related dotNet technologies). But slowly I have made a shift just out of pure interest and to broaden my horizons. Right now I work in a pharmaceutical company. And I am not working much on the dotNet framework or on anything related to C++/C#/ASP etc. Am working off SAS.

Let me explain a little. Since most of you reading this would be from the IT/ Software Dev/test background and not from the pharmaceutical side, it would be difficult to comprehend. But anyone who has heard of clinical trials will have a fair bit of idea of what is about to follow. Before a new drug comes out in the market, years are spent on R&D for this drug. Before its release in the market, it has to be approved by the FDA. To get the FDA's approval, clinical trials (tests) are conducted on a select group of subjects (the layman might call them patients). First the tests are conducted on animals, and after determining a certain level of safety, they are tested on humans (yes, each person who is being tested has to sign a consent form etc.). Finally these results are interpreted by applying certain statistical analysis and presented to the FDA. Here is an area where I come into the picture. All the data collected is firstly entered in the form of a table (called a SAS dataset, but you could compare it to any database table like ACCESS, SQL etc.). These tables are exact duplicates of the way data has been collected on paper for every subject. Lets take for example every subject will have demographic data, such as ID, sex, race, date of birth etc. So if there are 1000 subjects in the study, you have a table (or dataset) called DEMOG which has all the above demographic information for the 1000 subjects. So on, you have another 20-25 tables or more (depends from study to study) of the subject records which would include lab data, adverse events, vital signs, disposition, ECG, taken at different times as per the statistical analysis plan.

Okay, I went off track a little there. The point where I come in is data validation. First off I write programs to make sure the data entered is valid. Now, this might look easy, but trust me, its not. You have to go through the entire statistical analysis plan (call it the specs). Break it down into pieces, apply these pieces to the datasets and then put them together. If they give the same picture which the statistical plan does, half your job is done. Again, relating back to IT. Always go through the specs before you start coding or testing!! All of my team depends on the fact that the data which comes out to them is validated and correct. If they work on incorrect data, get their programs up and running and then we have to make changes to the datasets, you know how disastrous it could be! Hence I have to make sure that I (or for that matter anyone who works on the validation part) know the data & the Analysis Plan inside out. Its as good as knowing the specs inside out before you start coding/testing.

The next part of my job is to write programs which would perform statistical analysis on the data. Like lets say, finding the mean of the maximum dose levels administered to subjects who were above 55 years of age. Once again, this is a simple example. We can get much complicated with survival analysis, Chi-Square tests, regression models to name a few. Again there are different ways you could choose a certain population. But are you doing it the most efficient and correct way?

To round it up, after a lot of reviewing, these results are finally publised and sent to the FDA for approval. Thats the SLC for "shipping a drug" in a pharmaceutical industry. Looks simple? It is... but not as simple as you think it to be. And before you ask "Why?", I'll follow it up in Part-II. Till then... Apoorva Joshi highly appreciates your patience. ;-)