After a discussion about interviewing testers I started to rethink how I conduct interviews and I decided to MS Paint as an application to see how candidates approach testing.
Update 20200615 - Note, I no longer use MS Paint. I use the applications that we create and test in the team that I am looking to recruit someone. I want to make the interview process as close to the real work experience as possible. This was my first attempt at using a ‘real app’ an I have left it here for historical reasons.
A long time ago, I wrote my own little app for use during interviews. You can play with it if you like - it has many deliberately injected bugs - so no raising defect reports with me, but feel free to share any experiences that you have with it.
You can also find this in my Compendium of Testing Apps.
I mainly used this in team based exercises, and primarily to explore presupposition analysis as a method for testing.
How I used MS Paint
So, now to explain the reasons why, and how, I used MS Paint.
I had 2 charters:
- A “risk identification tour” charter
- A Scenario charter: “Function A” has changed - the existing automated tests have passed but it has a lot of internal dependencies - can you give it a quick once over and a quick regression of “sub function Y” and let us know what you find?
I picked paint for a number of reasons:
- I class it as a ubiquitous Windows application
- The fact that everyone has seen it should trigger some ‘assumptions’ about it that the process of testing can help challenge
- Since I classed it as a ‘mature’ application, my immediate assumption was “will I even find any problems in this application, MS must have made it really stable by now!” (I do seem so naive and overly-optimistic sometimes). That initial moment of panic seems like a good trigger to put people in a non-complacent state and view the application hyper-critically and focus their attention on their own test approach.
- I want to see the candidate’s initial ‘tour’ of the application to find out what areas they identify as having high ‘risk’, and their rationale for that identification
- Paint really does have bugs in it - lots of bugs - at least it does if your ‘tour’ brings you to the same risk area that I looked at - you can probably find bugs in the rest of the app too (my naivety got knocked out of me after 2 minutes of testing)
- I’ve seen MS Word and MS Excel used in a lot of testing demos but not MS Paint, so testers may not already know its flaws (I’m trying to keep this blog post deliberately vague so that I don’t highlight all the ones that I found, or even the areas I looked at)
- It has a lot of Oracle challenges - i.e. how do I know if I saw a correct response to my action? So should make for a good question and answer session
- It seems like a good spring board for tooling questions - e.g. what tools could have helped you during your testing?
But to help me provide some interview variety…
- What applications do you set your candidate loose on,
- What charters do you give them?