A book review of the The Craft of Sofware Testing, a book with a lot of good advice buried, suitable for Code Level Workers.
Not easy to read but still worth it
- Author: Brian Marick
- Publisher: Prentice Hall
When the author, Brian Marick, describes his own book as “somewhat dated…written in a less spritely manner than I’d use today… this is not how I do things today”, that doesn’t really add up to a particularly motivating sales pitch for the book.
My copy has the appearance of “printed on demand” - which has resulted in a slightly wonky copy, but at least the book remains in print.
I think it that anyone reading the book will see how Brian ended up as one of the signatories of the Agile Manifesto. I found that I read the approach, that Brian outlines for Test Requirements as mirroring the approach I use when writing test ideas and analysing Agile stories. I believe that much of the text could find ready application to people working with Agile Stories - but I think the reader will get bogged down by the examples on first reading.
As I read the book I found myself wishing that it had undergone a re-edit in the years since its publication. I found it very heavy on the examples, and the important advice that the example text contains struggled to stick out. The very few ‘summary’ sections makes it a hard book to skim or revisit quickly to remind yourself of the main points. So sadly the good advice remains buried in the text. I found all the asides, hints and tips and ’thinking aloud’ the most useful content in the book, but these sections do not form the ‘meat’ of the book - the examples form the meat, and they seemed a slog to read.
I had a quick look around Brian’s website to see if the important elements from the book had made it there and I found a few papers that mirror the book (all available at http://exampler.com/testing-com/writings.html):
- A Manager’s Guide to Evaluating Test Suites
- The Tester’s Triad: Bug, Product, User
- Faults of Omission
- How to Misuse Code Coverage
- Notes on Object-Oriented Testing, Part 1
- Notes on Object-Oriented Testing, Part 2
Also read the “Testing For Programmers” course notes on the web page - visit the bottom of the ‘writings’ page for the links.
Unfortunately none of these papers deal with the important concepts in the book of “clues” and “testing requirements”.
You can download Appendix B, the Generic Test Requirements Catalog, from Brian’s web site but unfortunately I could not find the more useful Appendix F available for download. Appendix F summarised the basic elements of the book and some of the thoughts I found useful, but I will still have to go back and give the book a 2nd read to extract full value from the text.
Interestingly the ’tester’ came number 2 on the list of targeted audience members, which certainly in 1995 made the book unusual. I suspect that the presence of code in the book will put many testers off. I found that all the code made the book unnecessarily ‘hard’ to pull out the really useful information, I think I would have preferred a ‘box out’ approach to the discussion text.
The approach of the book seemed well summarized on page 12:
“…I build the test requirements checklist alongside the design, and I constantly check the design against those requirements. When the design is finished, I build at least some of the test specifications. I expect that combining the test requirements and choosing specific inputs will job my imagination into noticing problems with the subsystem’s design. After they are fixed, and the test requirements and specification are updated, I think in some detail about how the design would handle each test. That sometimes finds bugs; at the least, it’s good preparation for writing the code. Then I implement the subsystem and the tests, trying always to write tests as soon as possible.”
This quote highlights the notion of a “test requirement” which represents a ’test idea’ for ‘how to try and find a fault’. Brian describes “clues” as the derivation source for test requirements. A “clue” described as something that ’needs to be tested’ which the tester picked up (from reading specifications, studying the application, general communication, etc.) .
A test requirements catalogue reads as an outline of ’test ideas’.
I think that this relates really well to the approach I adopt when working on Agile stories and identifying the acceptance criteria and tests required. I try and teach this approach to testers and analysts on the project. I don’t really want to send them to this book just for that information, since I’d also have to create a reading plan to target them into the ‘right’ bits of information, but I haven’t seen a simple write up of ’test requirements’ on Brian’s web site to point them at instead.
A few hints and tips that I pulled out of the text when reading the book:
- gain an awareness of the types of problems that your test approach will not find
- Review path coverage after the test analysis, rather than driving test analysis from path coverage
- Create tests by predicting faults - using general rules abstracted from common errors
- Errors with test requirements: overly general, too small (i.e. missing requirements)
- Use missing code coverage as a pointer to missing clues
You can extract value from reading the worked examples but you may have to slog through them a little - they do have little ‘asides’ about the thought processes in use, and the final write up showing the requirements catalogue against the specification and thought process provides useful examples of how to build up ’test ideas’ quickly.
I find myself tempted to not recommend the book because it does feel like a slog to read, but since I want it on my bookshelf, and I want to read it in more detail and actually slog through the examples in more detail because I think I missed some nuggets of useful information. I think that all means, despite its faults, I end up recommending it with caveats…
- If you work on Agile projects and you are are prepared to slog through the examples then I think you’ll find value
- For most general testers, or testers on waterfall projects, or testers without an interest in code and working through the examples in a book - I recommend that you steer clear, this book will probably not work for you.
The examples do feel dated and I would not call the text ‘spritely’, and you probably wouldn’t do testing like this, but if you can mine the text then you will find value here.
Related Links
- StickyMinds book review
- Brian Marick’s Websites: Agile and blog
You will need a Github account to comment. Or you can contact me with your comment.
I reserve the right to delete spam comments e.g. if your comment adds no value and its purpose is simply to create a backlink to another site offering training, or courses, etc.