MCA 317

Software Testing Notes
I think in many ways the more recent revolution in how software is created comes not from programming languages or code organization but from how testing and quality assurance are accomplished. We learned a decade or so ago how an object-oriented approach helps in producing good software. To go further a testing structure is neccessary.
Many companies only have "black-box" testing done by non-programmers who are part of a "Quality Assurance" or "Quality Engineering" department, which is separated from the department which programmers are in. Their testing consists of using the program as an end-user would, while carefully tracking the results.
There are also various forms of automated "black-box" testing (i.e. external programs which simulate the actions of a user) as well as "white-box" testing which contacts internal pieces of the program not normally accessible to the end user.
Unit Testing
Unit testing is the practice of writing small automated test for the functionality of self-contained units within the code -- generally a single class or small set of classes.
For example, an Enterprise Java Bean which accesses a database might have a unit test which checks to make sure that basic get and put functions work.
JUnit is a set of Java classes that aid in unit testing. However, formal tools are not required. A simple convention is to include a main method for a Java class, which instantiates and tests some simple functionality of that class. Additional information can be found in the following articles:
JUnit Test Infected
Detailed Unit Testing Code Example
Automated Development Tests
A set of tests should be bundled together as a minimum baseline for interim development versions. Ideally, this should be run as a single step after compiling, as a simple make test or ant test command.
This can be as little as a handful of the unit tests discussed above. As a project approaches the point of hand-off to the QA department, the tests should probably be more extensive and establish at least a few of the minimum standards needed for hand-off (as described in the QA guidelines).
Web projects should develop simple automated testing as well for link testing and so forth. General use tools (such as a linkbot) should be made available within the company.
Logging
Logging is a neccessary process for debugging, testing, and maintenance. Like unit tests, log and info messages should be seen as an integral part of the product. They should not be considered as temporary code hacks to be thrown away later.
For any large-scale projects (and many others), logging procedure should be multi-level and runtime-configurable. These mean:
Multi-level means that there should be several distinct modes of logging for different situations, as opposed to simply "on-off". A typical arrangement of levels would be:
"FATAL" designates very severe error events that will presumably lead the application to abort.
"ERROR" designates error events that might still allow the application to continue running.
"WARN" designates potentially harmful situations.
"INFO" designates informational messages that highlight the progress of the application at coarse-grained level.
"DEBUG" designates fine-grained informational events that are most useful to debug an application.
Runtime-configurable means that the level of logging can be changed without altering the code. This can be handled by a variety of means such as command-line options, a configuration file, or environment variables. This feature is vital for debugging and testing of a versioned, stable product.
Logging with these features is a major step over the common practice of the programmer adding in "print" lines and then commenting them out when not needed. The most important change is that the programmer should look at log statements as a permanent part of the program rather than temporary hacks. This greatly aids the debugging process as well as use by external administrators.
These features can be implemented by custom code or by using an existing logging package. For Java, the open-source project log4j is an example of a package which supports these features with minimal impact on processing speed.
Bug Tracking
Once a software product is in the pipeline towards being made public, it frequently goes into a formal process where specific bugs are numbered and tracked. One of the pitfalls of such systems is that they can stifle open communication by being to formalized. In developing a product, the programmers need to keep others informed -- especially in communicating an accurate timeline of when fixes will be complete.
In reality, not all of the intended product requirements are always going to be met. Too often, engineers work under increasing pressure to get requirements completed by the deadline. Desperate to meet all the requirements, they work furiously right up to the deadline, but often what is produced is not their best work. Then, when the product is turned over QA testers, it is full of bugs which cannot be fixed in time for the intended shipping date, thereby resulting in either a missed product launch date or a buggy product.
The lesson is that programmers need to assess as early as possible which goals are realistic. At that point, either the requirements or the goals need to change, and other teams informed of these decisions.
Learn to program at Brooks College, San Jose