The Complete Guide to Software Testing - Book Review

A book review of The Complete Guide to Software Testing, suitable for Anyone who tests.

Model Based Testing for Beginners

• Authors: Bill Hetzel
• Published: 1988
• Page Count: 293

Table of Contents Overview:

• Preface
• Part 1: The Testing Discipline
• 1 An Introduction
• 2 Principles of Testing
• 3 Methodology
• Part 2: Testing Methods and Tools
• 4 Testing through Reviews
• 5 Testing Requirements
• 6 Testing Designs
• 7 Testing Programs - Testing in the small
• 8 Testing systems - Testing in the large
• 9 Testing Software Changes
• 10 Testing Software Packages
• Part 3: Managing the testing function
• 11 The role of management
• 12 Organizing for testing function
• 13 Controlling the testing function
• 14 Putting the pieces together
• Appendix A: Software Test Practises survey
• Appendix B: Sample Testing Policy
• Appendix C: Quality Measurement Diagnostic Checklist
• Testing References
• Index

When reading this book I was reminded just how much the position of testing in the development process has changed relative its position at the date of publication. Testing is now more recognised as a discipline within the development process that requires specialist skills and specialist testers, organisations are more willing to set up independent test teams. And although we still struggle with many of the identified problems, which is partly what makes the text still relevant, the situation for testing has improved. But the situation has not yet improved far enough and there are still lessons to learn from this book.

We are led through the testing process in typical fare by introducing definitions of testing and quality but ultimately the pragmatic is stressed. Testing is “a necessary information gathering activity to enable us to evaluate our work effectively.”

The basic principles and possible misconceptions of testing are explored; complete testing is not possible, testing is creative and difficult, testing occurs throughout the life cycle to help prevent defects from propagating, testing is risk based, testing must be planned. All of this sets us up for the rest of the book and to view testing as a quantifiable approach with tools, techniques and methods rather than an art.

A simple methodology called STEP (Systematic Test and Evaluation Process) is presented with useful process diagrams showing flows and the position in the life cycle of various standard forms of documentation including test plans, test specifications, and error logs.

Part 2 focuses on what it takes to make testing happen throughout the whole development process, “it should not surprise us that what it takes to build good software is also what it takes to build good test sets”.

Development process integration is stressed through the text, hence our first test techniques are reviews which are conducted by everyone involved in the process. Effective review processes, although they take practice can help make the staff involved on a project realise that testing is their responsibility too, and that they already do testing.

The mind set that “testing is not my job, it is the job of the tester” is prevalent on development projects and one difficulty, should you choose to reframe reviews as a testing process, is that they may resist and cease to do reviews. This mind set may actually be more frequently encountered now that testing specialists are involved in projects. Although the text does address this conundrum in chapter 12 through effective communication and management, as that is essentially the solution to this problem.

I was surprised to notice on re-reading for this review that, when requirements are reviewed, we are recommended to review acceptance tests alongside them. I have never encountered an organisation that does this. Obviously a very valid approach, the writing of tests can highlight many defects, and a practise adopted as part of the main cannon of Extreme Programming techniques.

Readers are pointed towards Weinberg and Freedman’s “Walkthroughs, requirements and technical reviews” for further reading. I too would recommend this and most of Gerald Weinberg’s other texts.

Requirements and designs are covered with an emphasis on doing them right using reviews and through cross-referencing them to the derived tests.

Hetzel classifies testing as being “in the small” and “in the large”. I assume that it was Hetzel that originated these terms as this is the first place I saw them, but I have seen them crop up again in various other computing texts, most recently Scott Ambler’s “The Object Primer”. This is an influential text.

“Testing in the small” deals with testing focused on the lowest levels of the software, essentially unit testing and is worth reading by developers and testers. A very telling statement on software practises is that “simply asking for test designs improves unit testing more than the adoption of any technique”. This variant of the Hawthorne effect does work in the real world and I would recommend it to any tester or development manager.

“Testing in the large” focuses on the higher more integrated phases of software testing, including system and acceptance testing. Here the planning of testing is quite rightly stressed.

Part 3 deals with the broader parts of testing; the management, control and communication of testing.

Hetzel likes self-evaluation and provides a set of thirteen questions that every test manager should ask themselves as to their effective adoption of the 5Ms; Management, Motivation, Methodology, Mechanisation and Measurement.

There are a variety of checklists and evaluations presented in the book that are worth returning too from time to time on a project to ensure that you are doing the right thing (see also Appendix C).

Testing specialists also have five key skills to consider, the 5Cs, Controlled, Competent, Critical, Comprehensive and Considerate.

I would encourage all software development managers to consider the following statement. “Hiring a testing specialist is a significant leadership action; it signals the manager’s interest in better testing and establishes clear accountabilities. In my experience, it is an excellent first step toward achieving a balanced improvement program”. Hiring testers is a good thing and should obviously done alongside an effective testing policy. Should any manager be required to justify their decisions to the board then there are helpful hints contained in chapter 12 as to how to do this.

The questions of ‘which metrics to produce?’ and ‘which documents to write?’ are one focus in Chapter 13. Recommendations are also made for improving the visibility of testing and thereby increasing its importance in the development approach, for example by posting the metrics where people can see them and making the metrics simple to understand graphs.

The last chapter once again stresses that the place of testing in the lifecycle is throughout it, at every stage, and conducted thoroughly and well.

I initially read this book over 10 years ago and had mentally filed it away as a classic but gentle introduction to testing that had been superseded by more modern and relevant texts. Re-reading it for this review has changed my opinion. This is a well-written and justifiably termed classic with enormous relevance for today’s development organisation.