18 - Case Study 5 - The Mathinator

By the end of this chapter, the reader will be able to:

• apply testing strategies to a complex applet

In the two previous chapters we looked at how to test programs for defects, and how to debug them when we find them. We also looked at exception handling, and how Java's exception system allows for robust programs to be written even in the face of the overwhelming odds stacked against us as poor, defenceless programmers.

In this case study, we're going to look at a scenario where we must debug someone else's code - the processes we go through for this are exactly the same as the ones we should go through with our own code. We'll look at how we show the existence of bugs, how we locate them once we have, and how we fix them.

For the many sins you committed in a past life, Shoddy Code In have chosen you to test their new flagship project: The Mathinator. The Mathinator is an applet that is planned for inclusion on an educational website for primary school maths students. Shoddy Code In have given you the task of testing the program to make sure it works.

Since no-one at the company is interested in maintenance (they have already started on their new project: an anthropology teaching tool called The Evolution Revolution), they have instructed you to simply fix any problems you come across.

The CEO of the company has stressed that you are not to interfere with the basic structure of the applet, but that you should try and catch any exceptions you find and simply indicate via a message box that an error has occurred. He has also stressed that the fault in such cases is with the user and not with the applet, so meaningful error messages should be provided so the user can avoid their shocking incompetence in future dealings with the applet.

We are given three files for this applet. One is an HTML file (obviously), one is the applet file itself, and the other is a class that holds the calculations to be performed.

public class Calculations { public int findLowest (int[] numbers) { int currentLowest = 0; for (int num : numbers.length) { if (num < currentLowest) { currentLowest = num; } } return currentLowest; } public int findHighest (int[] numbers) { int currentHighest = 0; for (int num : numbers.length) { if (num < currentHighest) { currentHighest = num; } } return currentHighest; } public int findAverage (int[] numbers) { int total = 0; for (int num: numbers) { total = total + num; } return total/numbers.length; } public int findMedian (int[] numbers) { return numbers[numbers.length/2]; } }

import javax.swing.*; import javax.swing.event.*; import java.awt.*; import java.awt.event.*; public class ExceptionsBlock extends JApplet implements ActionListener { int numbers[]; JButton add; JButton remove; JButton average; JButton highest; JButton lowest; JButton median; JTextField entry; Calculations myCalc; int counter = 0; int averagen = 0; int highestn= 0; int lowestn= 0; int mediann = 0; public void init() { myCalc = new Calculations(); setLayout (new FlowLayout()); numbers = new int[10]; add = new JButton ("Add"); add (add); add.addActionListener (this); remove = new JButton ("Remove Last"); add (remove); remove.addActionListener (this); average = new JButton ("Average"); add (average); average.addActionListener (this); highest = new JButton ("Highest"); add (highest); highest.addActionListener (this); lowest = new JButton ("Lowest"); add (lowest); lowest.addActionListener (this); median = new JButton ("Median"); add (median); median.addActionListener (this); entry = new JTextField (10); add (entry); } public void paint (Graphics g) { super.paint (g); for (int i = 0; i < numbers.length; i++) { g.drawString ("" + (i + 1) + ": " + numbers[i], 30, 60 + (i * 20)); } g.drawString ("Average: " + averagen, 150, 120); g.drawString ("Highest: " + highestn, 150, 140); g.drawString ("Lowest: " + lowestn, 150, 160); g.drawString ("Median: " + mediann, 150, 180); } public void actionPerformed (ActionEvent e) { int f = 0; String num; num = entry.getText(); if (e.getSource() == add) { f = Integer.parseInt (num); numbers[counter] = f; counter++; } if (e.getSource() == remove) { counter--; numbers[counter] = 0; } if (e.getSource() == average) { averagen = myCalfindAverage (numbers); } if (e.getSource() == highest) { highestn = myCalfindHighest (numbers); } if (e.getSource() == lowest) { lowestn = myCalfindLowest (numbers); } if (e.getSource() == median); { mediann = myCalfindMedian (numbers); } repaint(); } }

The Mathinator is a very simple applet - it has a number of buttons that perform actions, and a display that prints out the state of a ten element array. It also has a text field so people can enter their values into the array.

Add and remove last are the interfaces through which people can change the state of the array. There are four other buttons that perform calculations on the array and display the average, the highest, the lowest, and the median values:

Fig 18.1: Mathinator Applet

This is a simple example of how complex testing can be. If we only had an applet to worry about, then we know where any problem is going occur - in the applet. However, we have an applet that is the view and controller, and another class that is the model.

If we press the button to compute the average and it displays a wrong number, where is the problem?

• In the button? Perhaps it's calling the wrong function.
• In the calculate method? Perhaps the code is wrong in some way.
• In the display? Perhaps it's getting the right answers but printing out the wrong ones.

In a piece of code with more classes, this problem becomes even more complicated, until the necessity of a formal testing strategy is overwhelmingly obvious.

The first thing we're going to do is unit test each of the methods in our calculations class through black box testing. We need a testing harness for this, so let's write one - a simple console based free standing application that makes use of our Calculations object:

public class TestingHarness { public static void main (String args[]) { Calculations myCalc; int[] input = { 1, 2, 3 }; int output; myCalc = new Calculations(); output = myCalfindHighest (input); System.out.println ("Output is " + output); } }

Our input in this case is an array of integers. First we're going to use black box testing to input our test cases, and compare our expected output to our actual output.

There are boundaries in this program, mainly at the extremes of the size of the array. The applet provides a ten element array as standard, so we need to test with arrays of size 0, arrays of size 10, and also arrays of size 1 and 6. These will be our boundary checks.

We have three main equivalence classes - arrays of all positive numbers, arrays of all negative numbers, and arrays that are a mix. However, since this is an array we have some other non-obvious equivalence classes - arrays that are in proper order, arrays that are not, and arrays that are semi-sorted.

So, first we need to write some test cases. To simplify this for the time being, we'll look at only positive numbers:

Fig 18.2: All Positive Test Cases

That's quite a few test cases - already we're getting into the territory when it's going to be cumbersome to manually input these tests. So we need to expand our testing harness to let us automate this. Let's use a comma delimited string to represent our array values, an ArrayList of these strings for our set of test cases, and then we tokenise the appropriate string into an array for input. See chapter six (String Theory) for a refresher of this topic:

import java.util.*; public class TestingHarness { public static void main (String args[]) { Calculations myCalc; int[] input = { }; int output; int counter; ArrayList < String > myTestCases; StringTokenizer myTokens; String tempString, temp; myTestCases = new ArrayList < String >(); myTestCases.add ("0"); myTestCases.add ("1, 2, 3"); myTestCases.add ("3, 2, 1"); myTestCases.add ("1, 2, 4, 7, 2, 4, 5, 8, 5, 10"); myTestCases.add ("10, 3, 4, 1, 2, 3, 7, 5, 3, 3"); myTestCases.add ("4, 11, 8"); for (int i = 0; i < myTestCases.size(); i++) { counter = 0; tempString = myTestCases.get (i); myTokens = new StringTokenizer (tempString, ", "); input = new int[myTokens.countTokens() ]; while (myTokens.hasMoreTokens()) { temp = (String) myTokens.nextToken(); input[counter] = Integer.parseInt (temp); counter += 1; } myCalc = new Calculations(); output = myCalfindHighest (input); System.out.println ("For " + tempString + ", output is " + output); } } }

Hrm - that looks quite complex! It's complex enough to warrant testing, in fact. We won't look at this kind of meta-testing in this chapter, but yes, we should write a testing harness for our testing harness to make sure that our test cases are being properly passed into our unit. Take my word for it that this one works.

We add our test cases to the ArrayList:

myTestCases.add ("0"); myTestCases.add ("1, 2, 3"); myTestCases.add ("3, 2, 1"); myTestCases.add ("1, 2, 4, 7, 2, 4, 5, 8, 5, 10"); myTestCases.add ("10, 3, 4, 1, 2, 3, 7, 5, 3, 3"); myTestCases.add ("4, 11, 8");

And then run the application to get our results for the findHighest method:

For 0, output is 0 
 For 1,2,3, output is 0 
 For 3,2,1, output is 0 
 For 3,1,2, output is 0 
 For 1,2,4,7,2,4,5,8,5,10, output is 0 
 For 10,3,4,1,2,3,7,5,3,3, output is 0 
 For 4,11,8, output is 0 

Say! That isn't even a little bit right!

So, we fill in our test table:

Fig 18.3: All Positive Test Cases and results

These were good tests - they indicated a defect in the program. Let's try with some negative array test cases. We'll use the same numbers, just change the sign.

Fig 18.4: All Negative Test Cases and results

Run this, and we get:

For 0, output is 0 
 For -1,-2,-3, output is -3 
 For -3,-2,-1, output is -3 
 For -3,-1,-2, output is -3 
 For -1,-2,-4,-7,-2,-4,-5,-8,-5,-10, output is -10 
 For -10,-3,-4,-1,-2,-3,-7,-5,-3,-3, output is -10 
 For -4,-11,-8, output is -11 
 

Hrm, mystifying! It seems to be finding the smallest number in all of these cases, but always finding zero in the positive cases.

Fig 18.5: All Negative Test Cases and results

Okay, we have one more equivalence class to check - mixing positive and negative numbers:

Fig 18.6: Mixed Test Cases and results

For 0, output is 0 
 For -1,2,-3, output is -3 
 For 3,-2,1, output is -2 
 For -3, 1, -2, output is -3 
 For -1,2,-4,-7,-2,4,-5,8,-5,-10, output is -10 
 For -10,-3,4,-1,-2,3,-7,-5,3,-3, output is -10 
 For -4, 11, -8, output is -8 

Curiouser and curiouser!

So, we're sure that our method is incorrect - we've found a bug and we know (roughly) where it is - let's look at our unit:

public int findHighest (int[] numbers) { int currentHighest = 0; for (int num: numbers) { if (num < currentHighest) { currentHighest = num; } } return currentHighest; }

So, let's consider our first problem, that we're always getting zeroes, even if there are no zeroes in the array. Have a look at the first line of our method:

int currentHighest = 0;

You have to wonder what would happen if we changed that and tried a positive number test case:

int currentHighest = 1;

And we get:

For 0, output is 0 
 For 1,2,3, output is 1

Hrm... so in the case of positive numbers, it is always returning what we have set to currentHighest, and if there are negative numbers in the array then it returns the smallest number. Change it back to 0, but keep a note of that... it sounds important.

Let's have a look at the next line of code:

for (int num: numbers) {

Well, that's pretty simple and unambiguous - let's assume that's correct for now. We can always come back to it if more promising lines of inquiry show themselves to be fruitless.

Next line:

if (num < currentHighest) {

So, at each stage we're checking to see if the number in the array is less than the current highest. What happens if it is?

currentHighest = num;

It then replaces the value of currentHighest with the value of the array at that index. Let's think that through with a solid example of our '1,2,3' array.

We start off with currentHighest being 0. We then compare our first element in the array (1) with 0. If 1 is less than 0...

Ah-ha! It should be 'if 1 is greater than 0':

if (numbers[i] > currentHighest) {

Let's try our positive test cases again now that we've changed this line of code:

For 0, output is 0 
 For 1,2,3, output is 3 
 For 3,2,1, output is 3 
 For 3, 1, 2, output is 3 
 For 1,2,4,7,2,4,5,8,5,10, output is 10 
 For 10,3,4,1,2,3,7,5,3,3, output is 10 
 For 4, 11, 8, output is 11 
 

Success! Our method now passes all its test cases and we stamp a big 'approved' label across it.

Okay, now let's try our findLowest method. We'll use exactly the same test harness, we just change what method we're calling:

output = myCalfindLowest (input);

It's the same thing, just giving the lowest rather than the highest - so we'll use the test data that served us so well for our last method. For positive values:

Fig 18.7: Positive Test Cases and results

Wow, does anything work in this program?

If we try for negative values:

Fig 18.8: Negative Test Cases and results

It passes all of those...

And for mixed input:

Fig 18.9: Mixed Test Cases and results

Passes all of those too. So it's only arrays of all positive numbers that have a problem. Let's look at the method:

public int findLowest (int[] numbers) { int currentLowest = 0; for (int num : numbers.length) { if (num < currentLowest) { currentLowest = num; } } return currentLowest; }

The last time it was our comparison that was a problem, but this time the comparison is correct. It's still throwing up phantom zeroes though, and the only place we have them in the code is in our currentLowest variable. Last time, thinking through a comparison with a solid example helped us quite a bit, so let's try it again with the array '1,2,3':

currentLowest is zero. If zero is less than one...

Hey... if it's an array of positive numbers, then nothing in the array will ever be lower than zero! We can't check it against zero... we need to check it against a number that is definitely in the array. Let's just put the first number in the array into currentLowest at the start:

int currentLowest = numbers[0];

And lo and behold, it returns all the right values!

Okay, now onto out average method. Using all our previous test data, we find it works perfectly. Sorted!

Now, onto the findMedian. If you don't recall, the median is the middle most point in a list of numbers. In the case of there being two middle-most numbers, then it's the average of the sum of those two. The key point about a median is that it is the midpoint of a sorted list of numbers:

Let's use some different test data for this, just to be devils. Actually, we have a very good reason for choosing different sets of data - we're looking at different behaviour that is dependant on the size of the array. Arrays with an even number of elements have different behaviour than arrays with an odd number of elements - this is a different equivalence class:

Fig 18.10: New Test Cases and results

Well, that's a start - clearly it is not working properly for even numbered arrays. Let's look at the code:

public int findMedian (int[] numbers) { return numbers[numbers.length/2]; }

Well, no wonder! We don't have any code for dealing with that, so we'll need to add it:

public int findMedian (int[] numbers) { int mid; int median; mid = numbers.length/2; if (numbers.length % 2 == 0) { median = (numbers[mid] + numbers[mid -1]) /2; } else { median = numbers[mid]; } return median; }

Now it works for our test cases - but we have only used sorted arrays as input. Let's try mixing it up a little:

Fig 18.11: Unsorted Test Cases and results

So, it doesn't work for unsorted lists - we'll need to include some functionality to deal with that:

public int findMedian (int[] numbers) { int mid; int median; Arrays.sort (numbers); mid = numbers.length/2; if (numbers.length % 2 == 0) { median = (numbers[mid] + numbers[mid -1]) /2; } else { median = numbers[mid]; } return median; }

And now it works - mostly. The averaging of the medians rounds up rather than providing a decimal answer - whether this is a bug or not depends on the needs of the user. Fixing this is left as an exercise for the reader.

So, that's our unit testing all done! Our unit has now been tested with black box testing, and we can see that it conforms with the expected behaviour. Now it's time to look back at our applet and see what we need to fix with that.

Unit testing an event driven framework is somewhat different from testing a method - it is difficult (at this point) to write a testing harness that would work effectively. There is also much more complexity involved - we must not only take into account the action that causes an error, but the state of all the variables involved. This can be time consuming, so we will only look at a few examples of this.

The easiest way to do this kind of testing is to base tests on a particular event, and only record the state of the variables that relate to that particular piece of functionality. We must use white box testing to ensure that the right paths of execution are being followed with particular events. So let's add in some debugging statements to our actionPerformed so we can track the flow of execution. All we care about is the flow, so we'll comment out all the actual functionality:

public void actionPerformed (ActionEvent e) { System.out.println ("In actionPerformed."); if (e.getSource() == add) { System.out.println ("In add."); } if (e.getSource() == remove) { System.out.println ("In remove."); } if (e.getSource() == average) { System.out.println ("In average."); } if (e.getSource() == highest) { System.out.println ("In highest."); } if (e.getSource() == lowest) { System.out.println ("In lowest."); } if (e.getSource() == median); { System.out.println ("In median."); } System.out.println ("End of method."); } }

And then we write our test cases:

Fig 18.12: Applet Test Cases

We have no way to easily automate this at the moment, so we won't - we'll just press the buttons ourselves. Since it's a user interface it is somewhat easier to test all possibilities since they are only as plentiful as we provide.

Going through the motions of pressing the buttons provides us with the following output:

Fig 18.13: Applet Test Cases and Results

Hrm! That's not right... it's close, but not correct. Our white box test has shown that the flow of execution isn't quite right. In every case, the median println is being triggered. If we look at that particular if statement:

if (e.getSource() == median); { System.out.println ("In median."); }

Ah, there's our problem - a rogue semi-colon before the brace. Remove that, and all is well.

We should also test the add and remove buttons to ensure that the state of the array is being changed properly. Let's give that a try. We'll document our test data in terms of values we've entered for our add button:

Fig 18.14: Add Button Test Cases and Results

if (e.getSource() == add) { try { f = Integer.parseInt (num); numbers[counter] = f; counter++; } catch (ArrayIndexOutOfBoundsException ex) { JOptionPane.showMessageDialog (null, "Array is full. Remove first."); } }

Now for the remove button - we know the add button works, so we'll work from a starting state and see what happens after we press the remove button a specified number of times:

Fig 18.15: Remove Button Test Cases and Results

Okay, we know how to fix that too!

if (e.getSource() == remove) { try { counter--; numbers[counter] = 0; } catch (ArrayIndexOutOfBoundsException ex) { JOptionPane.showMessageDialog (null, "Nothing left to remove."); } }

Or do we?

Let's try some more tests to make sure:

Fig 18.16: Revised Remove Button Test Cases and Results

Yep, seems fine. Now let's try adding something again:

Fig 18.17: Additional Add Button Test Case

Say what? Now we're getting our error message when we catch an ArrayIndexOutOfBounds error message! Why is that?

Let's put in some debugging to see exactly what index we're trying to access - hopefully that will shed some light on the subject:

if (e.getSource() == add) { System.out.println ("Index is " + counter); try { f = Integer.parseInt (num); numbers[counter] = f; counter++; } catch (ArrayIndexOutOfBoundsException ex) { JOptionPane.showMessageDialog (null, "Array is full. Remove first."); } } if (e.getSource() == remove) { System.out.println ("Index is " + counter); BOLDtry { counter--; numbers[counter] = 0; } catch (ArrayIndexOutOfBoundsException ex) { JOptionPane.showMessageDialog (null, "Nothing left to remove."); } }

So, we go through the stages that we need to replicate our error:

Fig 18.18: Erroring Test Case

Followed by an add operation. The output to our console is:

Index is 2 
 Index is 1 
 Index is 0 
 Index is -1 

Ah-ha... so when we next press add we try to access index -1 to insert the new element. That's wrong... so our remove button is decreasing the index before it catches an exception:

try { counter--; numbers[counter] = 0; }

It's not the counter decrement that will cause an exception, so that line of code is always executed - it's the next line. Every single time we press remove we will decrement the counter by one, so we can put it into a state that means we can never add another element.

We have a catch statement that is designed to deal with our contingency handling code, so let's make sure our counter can never go below a valid index:

catch (ArrayIndexOutOfBoundsException ex) { JOptionPane.showMessageDialog (null, "Nothing left to remove."); counter = 0; }

And now it's working perfectly - we've unit tested our applet, and also unit tested our model class. Now we need to link the two together.

This should be easy! We know for a fact both pieces of code are working, so all we need to do is link them up and use our applet to supply some data and let the monies roll in.

Alas, no... it's not quite that easy:

Fig 18.19: More Damn Errors

Curses! What the hell is going on? We know these bits of code work - we've gone through them with our testing harness. Why aren't they working now?

Here is where the benefit of integration testing shines through - consider if we had just started entering numbers into the applet and trying to test it that way without testing each part individually first - we'd be lost as to where this error was coming from. At least this way, we know that it's not the applet, and we know that it's not the Calculations class, so it must be the messages that are being passed between them.

What are we passing? We're passing a ten element array of integers. But the way Java deals with instantiating an array of primitives is that they all get set to zero... so really, there's nothing wrong with the program. We're just not sending the information we thought we were. Instead of sending the array '1,2,3', we're actually sending the array '1,2,3,0,0,0,0,0,0,0'.

That's not what we want - we want to be able to set a cut-off point as to where searches and calculations should end. We don't want empty elements taken into account unless we explicitly entered them - so we need to also pass the number of elements we've entered into the array as parameters to the method calls.

Now, we really should go back to our testing harness, and change it so that we also enter a cut-off point for these methods. That's left as an exercise for the reader. For the purposes of this case study, we'll pretend we've done that and what we came up with for our calculations class was:

public class Calculations { public int findLowest (int[] numbers, int cutoff) { int currentLowest = numbers[0]; for (int i = 0; i < cutoff; i++) { if (numbers[i] < currentLowest) { currentLowest = numbers[i]; } } return currentLowest; } public int findHighest (int[] numbers, int cutoff) { int currentHighest = 0; for (int i = 0; i < cutoff; i++) { if (numbers[i] > currentHighest) { currentHighest = numbers[i]; } } return currentHighest; } public int findAverage (int[] numbers, int cutoff) { int total = 0; for (int i = 0; i < cutoff; i++) { total = total + numbers[i]; } return total/cutoff; } public int findMedian (int[] numbers, int cutoff) { int mid; int median; Arrays.sort (numbers); mid = cutoff/2; if (cutoff % 2 == 0) { median = (numbers[mid] + numbers[mid -1]) /2; } else { median = numbers[mid]; } return median; } }

And for our actionPerformed:

if (e.getSource() == average) { averagen = myCalfindAverage (numbers, counter); } if (e.getSource() == highest) { highestn = myCalfindHighest (numbers, counter); } if (e.getSource() == lowest) { lowestn = myCalfindLowest (numbers, counter); } if (e.getSource() == median) { mediann = myCalfindMedian (numbers, counter); }

Now we're getting the right answers, and we only have the unexpected behaviour of our findMedian method to deal with.

Remember how we talked about passing by value and by reference way back in the early chapters of this book? This is an example of the implications of this - when we sort the array in findMedian, we're not sorting our own copy of it, we're sorting the very same array that the applet is making use of. If we want to keep the state of the original, we have to make a copy and sort that:

public int findMedian (int[] numbers, int cutoff) { int mid; int median; int tempNumbers[]; tempNumbers = new int[cutoff]; for (int i = 0; i < cutoff; i++) { tempNumbers[i] = numbers[i]; } Arrays.sort ( tempNumbers ); mid = cutoff/2; if (cutoff % 2 == 0) { median = (tempNumbers[mid] + tempNumbers[mid -1]) /2; } else { median = tempNumbers[mid]; } return median; }

And that's it, we're done. Our applet works and should have no unexpected behaviour. The trials of our testing are over, and the CEO of Shoddy Code Inc. should be very happy with our work!

That may seem like a very time consuming process to go through, and that if you do this kind of stuff you'll spend hours performing all of this testing. Remember that we cut quite a few corners in this case study - testing in an inherently iterative process. If you make a change, you need to test it. If you fix a piece of code because of a test that revealed a defect, you need to test that fix also.

We used only black-box testing for our Calculations class - we should have also spent time using white-box testing. It's a laborious, time-consuming and thankless task.

As programmers, we often leave testing to the last minute, and treat it as an unavoidable chore that we want to get done with as soon as possible. This is why we end up with broken code that doesn't work properly - testing is vital, and should be one of your primary concerns when developing.

Most software companies have a dedicated team of testers - perhaps an in-house team or perhaps they outsource. Perhaps working through this particular case study will make you more inclined to treat them nicely should you ever have one return a piece of your code with a list of ten thousand errors. Just think, it could be you who has to find the errors!

Further Reading

The following table details further reading on the topic in this chapter, and also any external resources that you may find useful.