Skip to main content

Dec 17, 2019 - 3 minute read - Software Testing Exploratory Testing Evil Tester

How I Test

TLDR; I split my testing into small chunks with a high level aim, and a generic classification. I make notes. I emphasise evidence more than upfront planning.

I provided an online talk at Dev Fest 2019 for Bishkek group. This is a short summary with links to the slides and exploratory testing notes.

PDFs and Slides and more details about the talk are available here.

One Thing

One thing I’d like to see more of, is actual output from the testing that people perform.

The issue is, that most testing is on a project and therefore confidential.

The only way we can ‘see’ the testing people do is when they perform it on an open source application.

I have previously created examples of how I test on YouTube, these have the disadvantage that I’m often trying to ‘explain what I do’ as I do it. This generates more cognitive load and changes the way that I test.

Debriefs

For the Bishkek talk, I wanted to:

  • test an application
  • as close to possible as I do when on a project
  • make the same level of notes
  • demonstrate how I track sessions
  • explain how I come up with test ideas

To do that, I decided to use a debrief format, where I do testing, and then debrief it to analyse what I did, why I did it.

Session Types

Over the course of my testing I classified my Session Types as:

  • Install
  • Health Check
  • Planning
  • Recon / Modelling
  • Debrief
  • Coverage
  • Exploratory
  • Admin

This is not a complete list of session types I use when testing. This is the list of session types I used when performing this testing.

You do not have to use the same names as these, own your testing, and name them as appropriate for your and your environment.

My sessions usually started with a Planning session so I would know what I’m doing, and were followed by a Debrief session so I know what I did. Giving the following rough order.

Notes

When I make notes, I:

  • use a linear format
  • keep simple text notes
  • add timestamps to my notes
  • record data and actual observations
  • write down questions
  • identify todos
  • describe defects
  • etc.

e.g.

You can see more notes in the downloadable pdf and on the conference page.

Lessons Learned

I found this a useful exercise because it caused me to ‘name’ by session types which I had only ever done informally before.

We explore at all points in our testing.

I plan in short bursts, before I test - to the level I need to, to help me prioritise and perform the next action. And I make planning decisions as I test.

  • The more we are focused on coverage
    • the more we constrain our exploration to the Coverage.
  • The more we are focused on Exploration,
    • the more our coverage has to be reverse engineered from:
      • our logs,
      • evidence, and notes.

There are more lessons learned in the slides.

Resources

I expanded some of these as Patreon posts as I was testing.


You will need a Github account to comment. Or you can contact me with your comment.

I reserve the right to delete spam comments e.g. if your comment adds no value and its purpose is simply to create a backlink to another site offering training, or courses, or etc.