Lessons Learned in Software Testing

Lessons Learned in Software Testing contains lots of short essays which are well worth reading and thinking about.

Table of Contents Overview:

• Preface
• 1 - The Role of the Tester
• 2 - Thinking Like a Tester
• 3 - Testing Techniques
• 4 - Bug Advocacy
• 5 - Automating Testing
• 6 - Documenting Testing
• 7 - Interacting with Programmers
• 8 - Managing the Testing Project
• 9 - Managing the Testing Group
• 10 - Your Career in Software Testing
• 11 - Planning the Test Strategy
• Bibliography

Authors:

• Brett Pettichord
• Cem Kaner
• James Bach

Stichomancy is the practice of utilising synchronicity to guide your reading. I have a number of computer books on my bookshelf which are suitable for this highly entertaining pursuit, books by: Jackson [1], Weinberg [2][3], Constantine [4], Perry and Rice [5]; to list just a few. And to that list I can now add Lessons Learned in Software Testing by Kaner, Bach and Pettichord.

Being suited for Stichomancy is not the best prerequisite for a sit down and read sequentially book reading strategy. It is however a very good prerequisite for a long lasting, dip in and out book. These are very good volumes to remind you of concepts, techniques and to give you pause to catch your breath during times of pressure.

293 lessons are partitioned into 11 chapters. Is there a mystical significance to the prime 293 being used for the lesson count and further use of the prime number 11 for the chapters? Is there perhaps a chapter per disciple, excluding the treacherous Judas (where is the heretical chapter numbered 12, what dark secrets did it contain that it could not be brought before the paying public?). In order to escape Earth’s gravity a rocket must travel at over 11km per second, are we as testers being encouraged to shake off the shackles of traditional software development thinking and explode into the outer reaches of our potential? Such dalliances with numerical analysis of the number 11 serve us no real purpose, and indeed are laughable nonsense, but this is a book that encourages trains of thought beyond those of a normal testing book, although probably not quite as far beyond as those above.

There are lessons that overlap each other in the various chapters but they are all slightly different projections of the underlying model. Ideally there could be several partitioning schemes, as the lessons could well be grouped in numerous different ways to aid the investigative and thinking process, but that is left as an exercise in applied epistemology for the reader. Different indexed lists don’t seem to be the best way to present information in book form.

Curiously, the book is far bigger than expected as it expands in your head as you read it. I did occasionally, very occasionally, in fact I think it was just twice, find myself shaking my head, but that is only because I have never been in the situation where those lessons could have been used, and because in the situation when the lesson could have been applied I took a different approach. I did find myself thinking - “hmmm, what if?” - quite a lot.

In fact I wonder now, if I had had this book when I started, would I have had to learn as much through experience? More likely, I would been better prepared to face those experiences because I could draw upon the years of experience embodied in the text. Don’t learn the hard way, read the book and if you really want to make life hard then be reminded of the lessons the hard way.

It is difficult to pull the fundamental concepts out of this book but one of the most obvious is the notion of context dependence (I’m cheating because that is mentioned in the title). Every site is different, every project on every site is different, every tester is different, different techniques and approaches will apply.

Part of the reasons for having a book like lessons Learned is to provide some alternatives but more importantly to help explain to the reader how to identify more alternatives, and the book does explore what “Thinking like a tester” means. To that end the book delves into psychology, system thinking, philosophy, mathematics and scientific exploration.

There is a lot of good, practical, pragmatic and often heuristic advice on dealing with change, communication, relationships, modelling, pragmatism, the politics of power, relationships, bug reporting, test modelling, use of cross referencing matrices, combinatorial testing. A vast, vast range of topics are covered.

I’m going to single out chapter 11 as a very effective chapter that explores the thought process behind writing an effective test approach and the communication of that approach to the rest of the project. This is a far more useful presentation of test strategies than the typical “this is the IEEE standard for a test approach” presentation in other testing books. Rather than presenting well-worn standards in a standard way, the book explores, explains, thinks and justifies, then asks you to disagree.

How I laughed when the text suggested that retired executives might fancy doing a bit of testing to have a less stressful occupation. Testing has never struck me as being the most stress free of careers although we do have to do all in our power to make it such, and certainly to reduce the stress of any staff working for us.

In essence the book is a set of projections of a model of testing on to a variety of topological maps viewed from a multitude of angles with a number of analytical models which are then interpreted by the reader’s own cognitive strategies. And because these strategies change over time, the book holds up to repeated readings.

I look forward to a future hypertexted edition with a stichomantic interface to provide daily nuggets of wisdom that I can apply on my spiritual path of Test Analysis. In the meantime, this book is highly recommended.

Minor Lessons from the Compendium Developments’ School of Testing:

• Keep a notebook
• Learn from the projects you have been on by conducting your own retrospective reviews.
• Learn Visual Basic
• Don’t just learn a programming language, learn how to program
• Learn how to design software
• Use the web
• Read between the lines
• Keep learning
• Documented ambiguity is your friend - chase it down during testing.
• Documented specificity is your friend - chase around it during testing.

References

• [1] Jackson: Software Requirements and Specifications [amazon.uk] [amazon.com]
• [2] Weinberg: The secrets of consulting [amazon.uk] [amazon.com]
• [3] Weinberg: More secrets of consulting [amazon.uk] [amazon.com]
• [4] Constantine: Constantine on Peopleware [amazon.uk] [amazon.com]