A Short History of my ISEB Software Testing Certification involvement
Back in 2001? 2002? Back whenever I noticed the ISEB certification starting, I thought "Hmm... how strange, I wonder why they would want to do this".
I read an early draft of the syllabus online and thought "Well this seems fairly simple, but misses out a lot of stuff that I do in the real world, but what harm could it cause?"
After the unleashing of the foundation certification, and training and examining began in earnest, I remember reading a trade press advert that offered a "High paid career in testing after one week of training and gaining the ISEB certification" (or words to that effect). I still had enough idealism to feel personally affronted and feeling that 'what I did' had some how become demeaned.
In that moment I vowed to 'Save the world!', or at least the small part of it relating to Software Testing Certification.
Ultimately I failed. I relate my story here.
...But lords and knights and other noble and worthy men that con Latin but little, and have been beyond the sea, know and understand, if I say truth or no, and if I err in devising, for forgetting or else, that they may redress it and amend it. For things passed out of long time from a man's mind or from his sight, turn soon into forgetting; because that mind of man ne may not be comprehended ne withholden, for the frailty of mankind.
The Travels of Sir John Mandeville
Idealism
I had vowed (and failed) to 'Save the world!' on a number of occasions before:
- when I built my first CASE tool (a JSP diagrammer with a built in COBOL interpreter),
- I built an ER diagram of 'Testing' (to understand and finally get a handle on Software Testing),
- when I built a configurable entity management system (to model any testing process).
- when I generated test scripts from graph models
Introspection: Perhaps, this idealism and failed attempts at Super-heroism led to the development of the lampooning Evil Tester comic character?
At any rate - I resolved to engage in the certification process and change it for the better.
Infiltration
At one of the BCS Sigist meetings (Special Interest Group In Software Testing), a call went out for people to help with the examination process (writing questions, marking, etc.). My chance to infiltrate had appeared, so I applied...
...Only to receive a rejection on the basis of not having passed the foundation certificate.
You see I had let the certification process proceed apace such that ISEB had added an additional layer of certification, called the 'Practitioner'. And now to receive consideration of competence to allow me to mark and write questions for said 'Practitioner' exam, I had to have made the grade of 'foundation'.
So I duly went out to pass the ISEB Foundation. Eschewing any form of training for such a simple and easy test. I downloaded the syllabus and began preparing for a public sitting of the exam. (see my notes on how to prepare)
Sadly this was not my only motivation. The ISEB Certification had already taken hold on the UK Job market and I had already had my CV turned away from two job applications because I did not hold the certification. So driven by the 'fear' of never working again, I sullied myself, but justified my actions on the basis of 'Save the world!'.
Making the grade
I found sitting the exam a strange experience. The courses that people sit through in preparation for the exam must train them in such a black and white manner, that some of the ambiguous and vague questions seem obvious to them.
But for me, trained in the real world, having had my black and white orientation knocked out of me, I wanted to answer with a "well, it depends, do you do X or ...?". And that led me to choosing the wrong answer on at least one occasion because, and I confess this with a shamefully lowered head, I did not achieve 100% on the 'foundation'.
But still... I made it!
I passed!
I achieved certification!
I can describe myself as a tester!
I can now wave my bit of paper and say... "I, am a tester!" (I violated the General Semantics 'is' of identify for dramatic effect)
"Nah nah, ne nah nah! I got a bit of paper, you don't. I got a bit of paper, you don't." Cough, back to the seriousness.
A second Infiltration
Armed with my certification, I wrote a second letter requesting admittance to the hallowed realms of ISEB examination... and received an acceptance in return.
The ISEB certification community contains many 'big hitters' from the UK Software Testing world so it acted as a surprisingly useful networking opportunity. Although this post may result in the 'throwing away' of all those hard won networking moments, who knows?
I made a point of attending the exam panel meetings as much as possible. While I could have involved myself more remotely, I thought that I would stand a better chance of 'changing' things if I actually attended the meetings in person.
I had never written or marked any exam papers before so I expected to receive a little tutelage to prepare me for the coming travails. But as with many things, I learned by doing, and I learned by making mistakes.
On duty my vision dissipates
My duties included at least elements:
- I wrote exam questions - foundation and practitioner
- I marked practitioner exam papers
- I reviewed questions
All the judgments that we took as an exam panel used the mighty syllabus, as a constraining tool (to not go beyond it), and a driving tool (to cover it). That sanctified document which, if 'twere not in, then we could not ask thereon.
Changing the syllabus did not happen lightly and I do not remember a syllabus change driven by the exam panel. All changes occurred when the 'higher' panel decided. For two panels did exist - one which did the marking and examining, and one which engaged in more lofty pursuits; deciding what teaching should occur and who should teach it.
Very quickly my idealised vision of change dissipated. And I got down to the business of writing questions and marking exam papers.
On the art of writing exam questions
I did not master the art of writing multiple choice questions. They either read too easily, making the answer obvious to even the most unschooled and inexperienced candidate. Or required such a long description of the context underpinning the scenario I wanted the candidate to unambiguously address with a multiple choice answer.
I quickly gave up writing multiple choice questions.
I did not consider my attempts at writing the longer, essay based, 'Practitioner' questions a greater success. These questions took a long time to write, for a number of reasons:
- Trawling the syllabus to find areas of sufficient complexity to challenge the candidates thinking process
- Constraining the question to the limits of the syllabus
- Writing the question, then writing an example answer, and iterating around that process took a long time
And then the question became subject to a review from the exam panel. Some questions never made it into the exam and lay strewn on the exam panel floor - more likely the foundation questions as a huge pool of these already existed.
The shortage of questions resided solely in the Practitioner exam pool. So every question brought forward got bashed into shaped, tweaked and tucked by the panel - unless withdrawn by the writer.
I think I remember the butchering of one of my questions where the judgement passed on it meant that we combined it with a subquestion on the Belbin personality types - which personally, I abhor its inclusion in the syllabus. The inclusion of Belbin represented a far worse sin than the exclusion of 'exploratory testing' or 'model based testing'.
While as, an exam panel we did our best to understand the effect the question would have on the candidates, you only saw the results when the candidates tried to answer it.
On the decipherment of chicken scrawl
Candidates please... use your best writing. But sadly time pressure, and all that, typically means that the best analogy for reading of an exam answer paper became the extraction of meaning from a paper which chickens had run over with ink covered feet.
Marking these papers took a long time. Typically two hours (sometimes more) per paper. Then we met with our exam marking partner - for each paper had two markers. To examine variance between our grading. If the variance in question seemed high then we conferred to see what one marker had seen that the other had not. For example, had one marker managed to work out why the candidate kept writing the word 'chicken' in their answer, what did they really mean?
We tried to mark fairly and consistently, using the marking guidelines, and with our knowledge of the syllabus. This did result in disappointments where people who you felt should not have passed, did manage to pass.
At the end of the whole process we collated the marks together to see if we needed to apply weightings to the final marks. For example if 'everyone' failed a question and we deemed that question at fault then we might apply weightings to 'be fair' to the candidates.
Until I learn to accept my reward
It seemed that the only 'change' I could effect involved:
- commenting on questions that did not 'stress' the candidate well - although always constrained by the syllabus
- writing questions to test the candidate effectively - but again constrained by the syllabus
At one point, after a long marking session I examined my vision statement.
- Had I made any inroads into my goal of "Save the World!" - answer no.
- Had I at least received compensation for my actions - answer no. We could not consider the monetary remuneration for all this work as 'worth it'
- Instead, I helped to perpetuate a system I did not believe in, nor did I see it changing for the better.
- Instead the trainers and ISEB received the monetary rewards while I had managed to voluntarily enslave myself for their benefit, to the detriment of my own time.
I made the decision to leave - I forget exactly when, I think I engaged in this process for about two years.
A Few Lessons Learned
Simple lessons. Probably already learned. But obviously easy for me to forget.
- Do not perpetuate systems that you disagree with
- I don't know how to stop this certification industry, but I do know that I can starve it of my involvement and my support.
- Writing exam questions requires experience and training
- Part of an effective certification process must involve effective questioning and evaluation. I have my doubts as to the degree that the process, as it existed when I helped with it, met that requirement. Given that we received no training in, or had no experience of, writing or marking questions.
- The certifications do not represent an assessment of ability
- I read many exam papers. Very few of these 'Practitioners', exhibited in written form, a degree of knowledge sufficient that I would want to employ them.
- Having this stuff on your CV really does help
- Having 'past member of ISEB exam panel' on your CV helps. If you support the certification process then I encourage you to join the exam panel. That way you help to perpetuate a system that you support and you do your CV some favours.
- My hidden agenda in the encouragement: I hope that you experience a growing unease and distaste for the process the longer that you involve yourself in it.
- Keep your vision in mind and track progress regularly
- In the heat of battle, you can forget the ultimate strategy all too easily so keep this in mind and track your progress towards it.
- Always charge your full complement of expenses
- Since I didn't join the panel for the money, I sometimes didn't claim all the expenses or follow up demanding payment of all the money after writing a question. (At least this allows me to not feel too bad about taking their money - I didn't take enough of it)
Postscript
And thus, taking solace in my wretched rest, recording the time passed, I have fulfilled these things, and put them written in this book, as it would come into my mind...
The Travels of Sir John Mandeville
So, my involvement summarised, to the best I can currently remember.
If you have any reflections on your involvement then I would love to see you write about them:
- blog them, and ping this post,
- or leave a comment pointing at your reminiscence,
- or write them in as a comment here.
Excellent experience report.
My involvement: I strongly criticized the current crop of certifications (not just the ISEB/ISTQB travesties, but others like them) in a keynote speech at EuroSTAR 2007. (You can see the slides here: http://www.developsense.com/presentations/notyetcertified.pdf ) The speech generated a good deal of controversy. I believe that I earned the respect of many people who had not heard much about me until then. Some of these were, are, supporters of certification and many more were not. I suspect that I offended and irritated a bunch of people whose respect wouldn’t mean much to me. In this sense, these certification programs have been good for me.
At EuroSTAR, I also earned the ISEB’s first anti-certification. It says
It’s signed by Simon Adams, ISEB, December 2007. It even has a drawing of the ISEB logo on it. I am proud of my anti-certification, and hope others apply for it.
—Michael B.
Hi Michael,
Thanks for the comment.
Out of interest, how would one apply for an anti-certification certificate?
Thanks for an interesting and informative read.
You employ an interesting writing style that I find enjoyable - you must read a lot. Have you ever tried writing other than this blog?
Hi Mark,
All flattering comments automatically get approved - all other commentators please note.
I do read a lot and I have tried writing other than this blog.
I did have some fiction published in a short run mag. And I wrote the character descriptions in the computer game XS, these were subsequently turned into well acted vocal snippets for the in-game encyclopedia and which the critics duly, and universally, panned.
Since I was paid a pittance for my contribution, and had to purchase my own copy of the game rather than receive a free one at the launch party, a launch party to which I did not even receive an invitation… I don’t begrudge it that fate. I actually had to purchase my copy in the bargain bin of a computer game shop - oh the shame of it.
Dear evil tester
The truth is that the purpose of the ISTQB certification, and many other certifications is to make a lot of money. The truth is that people love to have pieces of paper that say they have achieved something. One of the most famous ISTQB teachers Dorothy Graham made a very interesting comment at EuroSTAR 2007 . She said: The foundation certificate is only there to scrape the bottom layer of ignorance away from the testers” Pretty hard words I must say. I believe she is correct. The foundation certificate exists to teach people the definitions of words and processes that a limited number of people have agreed are important to know. Any claim that anyone becomes a good tester by taking the exam is untrue. I teach a class called Software Test Design which is built on classical test techniques as well as my own experiences. No exam, no diplomas but a lot of discussions and exercises that hopefully will help students to solve real world problems. But it is not that easy to sell! Why, because whoever pays for the test classes believs that the ISTQB certificate is important to have but actually learning something useful is not!? So right now, in Sweden, the truth is that a lot of companies hiring testers require the certificate from candidates applying, consultancy companies use it as a selling argument. So while Michael Bolton actually wants to teach his students something useful, this is not the case with the certificate. However, ALL really good testers I have spoken too are totally aware of this. To really know if a tester is any good requires some knowledge and some additional work and that is obviuosly too much to ask for most…
OK, I could go on forever but suggest that anyone that is interested to read Michaels slides that he linked to.
“Dear Evil Tester” - I love that, but…mental note… I must make sure to personalise this blog so that people associate Evil Tester with the cartoon and not with me.
I second the point about reading the slides. I know the slides don’t convey the full spirit and drama as when Michael presented them, because I watched his talked at Eurostar. But hey ho if you missed it… you missed it. Read the slides.
Thanks Tobbe,
Alan
Compelling, organized, honest, and well-written. I confess I once had a similar notion to see what it would be like to take the exam and contribute from the inside so I, too, could have a shot at curing the world of bad certifications. I was on the fence until reading your report. I now feel cured of that desire, and consider you a tester who just ran a Build Verification Test on the ISTQB certification process and marked them with a FAIL, signalling to the rest of us reasonable, practical, heroic-minded folk that for us to try what you tried would be a colossal and disheartening waste of our time. Your blog entry here may be a force that changes the world in the way you truly intended, so thanks for your honorable service!
Thanks for the flattering comment Jon,
I think that if other people tried to do it the way I tried, at the level I participated, then they would fail in the same way.
Alan,
Point taken about certification …
How do we (people who view certification in its current form as good for the craft of testing) address folllowing social issues that arise out of our stand on certification (on the lines …”Well, let us agree for a moment that todays certifications are bad. What is the alternative, what solutions you suggest”)
1. New testers who are intimidated by certifications by recruiters, head hunters. They simply say “If I am not certified .. I will loose few good job offers”.
2. Managers of IT services companies (especially in India) say “our customers want certfied testers. I must provide them those people. Having x number of certified testers is a big boost for me against the compitition”
So it is new testers, customers who outsource(US and Europe) and IT test managers who are fueling the need for certified testers … if there is money to be made in this demand - people will make it.
How will the world of certification look from their angle?
Shrini
Hi Shrini,
Good Comment.
1) For the new tester. I say that they will probably lose some job offers. And they may lose ‘well paid’ job offers. But they will probably not lose any ‘good’ job offers, since those offers would come from companies that do not know how to recruit testers. I don’t think the industry tackles this problem by communicating to the tester, we do it by communicating to the hiring manager. So one solution here probably requires providing more information on how to effectively recruit testers and getting that information in the hands of people who currently rely on ‘lack of certification’ to weed out ‘bad’ testers.
2) In the short term this strategy makes sense for the offshore supplier. Having their testers certified allows them to wave that as a selling point to customers, in the same way that they used CMM Level 5. But eventually everyone has certified testers, and everyone has CMM Level 5. They very quickly lose their edge and become homogenized. A similar solution to above requires getting information out, to the people ‘buying’ the offshoring, on how to effectively choose an offshore supplier. The offshore suppliers themselves could help by ‘educating’ the purchaser on reasons why ‘certified testers’ do not mean ‘good testers’ and building a better sales pitch or value proposition.
I think that neither of these solutions will take, as ‘certification’ provides an ‘easy’ option.
Alan, thanks for that report. I applaud you for getting involved and trying to improve things. I’m curious to hear more about what kind of effort you think would be required to change the syllabus, and whether incremental changes to the syllabus would be worthwhile. Could it be that you’d have to be pretty happy with the syllabus as it is, and the certification process in general, in order to be willing to put in the effort to make any changes?
I’m lucky in the US that I can simply ignore the certifications and not tangibly hurt my career.
-Danny
Hi Danny, you have posed a really challenging question.
So challenging in fact, that I don’t have time to answer it at the moment.
I add this meagre excuse for an answer as a temporary measure until I can formulate a sensible reply, and so that other people can ponder your question as well.
… a few days later…
Unfortunately, I do not think that changes to the syllabus alone would address the problems I have with the certification process.
I do think the syllabus could change very quickly. If the people who set the syllabus decided to change it.
I could guess at reasons why it does not change quickly but pure conjecture on my part would only fuel a flame session and create an unprofitable chain of debate. I can’t even say that “the people who set the syllabus do not seem to believe that the certification process has flaws that require the syllabus to change”, because the syllabus has changed, and the syllabus has beget yet more syllabus.
I simply do not find that type of change appropriate nor do I find it desirable.
In order for me to put in effort to make changes to the syllabus I would have to believe that the certification process ‘worked’ and really did provide its advertised benefits, and currently I don’t.
I would rather put my efforts into writing on blogs like this, clarifying my thoughts and opening them up to criticism (although everyone commenting thus far has treated me very nicely), and making the subsequent writing available to anyone who stumbles across it on whatever learning journey they choose to engage in.
So I do not think that anything would drag me back to the certification process.
Alan,
I love your commentary on ISEB. I agree that its a a nice thing in principle but just will bring down the profession of testing. I loved your post so much so that I commented on it on my blog. http://blog.theautomatedtester.co.uk/
I hope that you enjoy the read and I hope that I removed all aspects of it being a rant. I did try quite hard on remove rant type wording.
David
Thanks for the comment David - I’ve added your blog to my rss feed. I think you managed to write a blog post that did not read like a rant. I don’t know how much editing you had to do to it, but if you feel passionately about the topic then I know how easily rant-like writing flows - I do try to avoid that style too.
As usual, entertaining and thought-provoking post. Reading it made me think about why I got one of those certs, and how my position about it shifted throughout the process. I also ended up with an interesting paradox.
My initial intention with getting the ISEB certification was (naively) to see if there were any areas of testing that I was missing in order to be “a good tester”. After reading the syllabus, and even more after the first day of the certification training, that objective changed to just having it on my resume as a door-opener for jobs (it was painfully obvious that in most of the subjects, my colleagues and me went much deeper than the training in our daily testing). I soon after discovered that, for the initial objective, it was much, much better to get actively involved in the testing community, which is much more productive, enriching and much more enjoyable!
However, having the ISEB was part of the reason I was offered my first job at this company. I am now the manager of the team I started in, and although I haven’t had the opportunity to hire someone, if I did, I would not make it a requirement (or even put it under “Desirable”) to have an ISEB certification for candidates. I didn’t either in my previous job, where I did hire people, and I was very clear about my reasons for not doing so (reasons already expressed in other comments in this blog). I have also been able to question the desirability of the ISEB certification from the day I started, in conversations with other managers responsible for hiring, and also the people in my team; I believe it has made an impact. I wouldn’t have been able to do this if I hadn’t go that job, which I got partially thanks to the ISEB certification.
I tend to see this as a different (maybe more passive) kind of attempt to bring the current certification system down. I’m not sure what the point of the certification is, but I do know that I don’t get better testers if they are certified by the current system, and I am not a better tester because of my ISEB. The only thing that nags at me is that this is a destructive attempt. I am not involved (yet, at any rate) in creating a certification system that does work. But I also wonder: is it needed? My ultimate certification is the quality of my work. Watch me test, and if it’s good enough for you, hire me. If it’s not, let me know why so I can learn, and maybe next time we’ll be a good match. Unfortunately, there’s still not many companies that actually ask you to test something during your interview. Maybe there should be a certification for “hiring testers”? :D
Hi Marta,
Thanks for describing your experiences, subversive communication tactics, and alternate strategies for learning.
I do not know how to create a certification system that works - one reason why I still haven’t written an answer to Danny’s comment.
I certainly think that a good training course in “hiring testers” could add value (I’d prefer to read the book) but then I would miss out on the certificate of completion.
Alan
Funnily enough, your google ads banner pulls up lots of ISTQB certification links.. I wonder what the click-through rate is for these :)
Hi Scott :) . Sadly not enough to compensate me for my time in the ISEB process.
An excellent read!
There is a lot of truth in what you say. BUT, when you work for a company that doesn’t believe in having a test team, with many project managers who don’t agree with putting aside project time for testing, you have to start to raise the profile of testing somehow.
Qualification of any sort is usually seen as professional, and often seems the only way to raise the profile of testers. If you are qualified (as the project managers and developers are in their areas) then they may take you more seriously.
This is the theory I am trying at the moment, and it is working. So, I have nothing better to use than what we have at the moment. If we can improve it then that would be just great…but my experience is that this is a big group who will be slow to change (and may resent any change being pushed upon them!)
Cheers Sarah
Hi Sarah, thanks for the comment. You have pointed out a sad truth in our UK industry.
A great read & brutally honest!
I’m involved in delivering ISTQB foundation training courses and I’m currently reviewing practitioner training course material. An experience in itself I must say! I have managed, until recently, to avoid teaching an ISTQB foundation courses purely because I have difficulties in staying out of the “real world” while teaching it. Not good if what you talk about is contrary to the Syllabus. I have to constantly remind myself to stick to the “script” or as I tell the participants, “when you walk through that classroom door, you left the real world and entered the ISTQB world”. Ironically, images of “Naria” pop into my head!
Do I disagree with certification and it’s intent? No! Why? Compare the topics and subjects that you cover at university & college. Does it guarantee you know at least something about “IT” in the real world? No, but it does tell me you should be aware of some of the basic fundamentals of technology & software development. Does it teach you how to apply them in the real world? Some would say yes, some no! My view is simply “it depends” because it isn’t black or white and while some people are able to translate skills learnt from University to there roles, others struggle!
To what level you know and understanding the “fundamentals of testing” only the interview process and a few well thought out questions will, at most, scrape enough paint off the surface to at least see that there’s no “wood rot” in that section of the wall!
While I’d like everybody to have an understanding of the “basics”, it’s the definition of what the basics are I have issue with! One classic problem for testing is around our testing vernacular. Get 10 testers in a room to work out how to perform Systems Integration Testing and they’ll spend 80% of the time arguing about it’s definition. Not exactly showing a consolidated approach and they wonder why the rest of the project team roll there eyes!
My opinion of the examination is that it is testing your ability to discern ambiguities in questions and very little else. Try and write multiple choice questions for the advanced ISTQB syllabus if you want a challenge. Unfortunately, “it depends” isn’t choice “C”!
Can you teach software testing? Yes! Does a 3 day course and an exam guarantee you’re a “good” software tester? No! If it did, we wouldn’t put all potential candidates for testing roles through a 30 minute exam which is focused on testing the candidates software testing skills, techniques and the basic knowledge we believe they should know. We use a practical set of real world scenarios, in which “it depends” is a valid answer as long as you can describe why “it depends”!
Keep those thought provoking blog entries coming, maybe the Testing Gods of ISEB/ISTQB will engage in discussions with those of us that “do the work”! Unfortunately, in my experience, those who live if “Ivory towers” are too high above the forest to know what happens on the forest floor!
Hi Shane,
Thanks for leaving your comment.
I found myself looking for the “it depends” answer when answering the exam questions. The Practitioner exam does try to address this - if the candidate has a good marker then that candidate should do well in the Practitioner exam with an “it depends” attitude, even though the candidate has to constrain themselves to also cover the syllabus and identified marking elements.
I have found that even a room full of ISEB certified testers can disagree about definitions. But other techniques exist for getting around that dilemma. So I’ll add that to my blog ideas list.
And I feel pleased that the blog post has started getting comments representing the perspective of people who feel more pro to the certification than anti, even if not 100% satisfied.
Thanks again,
Alan
Well written indeed. I will attempt to present a different perspective here.
As a rule, technologies, adoption of technologies, trends and practices in South Africa tend to be up to three years behind the UK and USA. The gap is getting smaller due to the internet and the global village phenomenon. In software testing, however, we are a long way behind. Which means that formal thinking and standardisation is not commonplace. I don’t know if it is because SA is a former colony of Britain, but UK standards seem to work well here, and naturally, the then ISEB certifications were brought to our country in 2001 by, who was at the time my employer, and one of the largest testing consultancies in the country. I wrote the Foundation in that year.
The benefits we gained is that:
1) at least testers started speaking in a common language (vocabulary :-)),
2) there is a way to distinguish between job applicants with some clue and the fly-by-night job hunters. (Unfortunately, there is a severe skill shortage here and people who can operate a mouse and keyboard are sometimes employed as “software testing specialists”.)
Although not as vocal as some others about the matter, I am a firm believer in context. I do not believe that certification makes a tester, but the fact that someone is capable of passing the ISEB exams tells something about their ability to learn about the discipline. I passed the Practitioner last year and it was difficult. The handful of certified practitioners here in SA are almost all in a test management/test consultant or more senior role, and are all worth their salt.
I also believe that the certifications have developed into a money-making scheme, which is sad. But I am not ashamed of my certifications. Instead, I am happy that I can function in a certification-sensitive context, if that is the particular context that I am working in at the time. You can’t criticise what you don’t know - which is why I really enjoyed your post.
Hi Walter,
I thank you for your kind words and appreciate the time you put into writing the comment.
Unfortunately, in the UK, the certification has not really allowed us to distinguish between applicants with a clue and fly-by-night job hunters, so I consider you more fortunate than I with that experience. I might feel differently about the certification if it had allowed me to distinguish between applicants in that way.
I think the early situation in the UK may have mirrored your experience - many of the early Practitioner candidates work in test management in senior roles and had a lot of experience.
I certainly don’t think anyone should feel any shame regarding an award of a certification that they worked and studied for. Social psychologists have studied the effects of displaying and promoting the certifications and awards that people have received and prominent display of certificates does have the general effect of promoting ‘authority’. (source “Yes!” Cialdini et al)
Thanks for sharing your experiences and I hope you continue to find value in the certification process,
Alan
As a fairly new tester at an established software startup (oxymoron?), I am the sole person responsible for pure testing, as a result I’ve been doing my best to cram information down my mental gullet and become what I need to be in order to do my job as best I can. I was hired based on prior experience in software and it was under the assumption that I could learn to test, certainly no one ever asked me for credentials in testing. So essentially, I could care less about an official stamp of approval from some certifying board who takes my money and gives me a piece of paper.
What I want, is to know enough that I can feel I’m up to my responsibilities. I just started to read your blog, James Bach’s blog, and I’m now looking at Cem Kaner’s videos, I’m starting to look for the books you all mention as being critical to your knowledge and understanding of the testing process and the basic principles that underlie it.
I just read Michael Bolton’s slides and I am impressed, and it also causes me to ask the following question. If people at the top of the testing community in terms of knowledge and intra disciplinary respect and recognition do not believe in a pay for a piece of paper style of certification, why isn’t there a network of professionals at that level performing an underground certification. Say, an online database of professional testers who have contributed sufficiently to the test community and routinely demonstrated their expertise such that they could say, here is an organization that I belong to that I believe in.
I don’t know if this question has been asked before, I would expect that it has and I simply by virtue of being new to the community have never seen the answer.
Hi Ed,
Reading between the lines you seem to have a great attitude towards learning software testing. And in managing to find Cem Kaner’s video early, you have found a great resource. Networks of professionals do exist so if you haven’t found the context-driven software testing yahoo group I encourage you to join, if the remit that defines it resonates with you. Other yahoo groups exist, as do other online communities.
Physical communities also exist, so Google your local area for software testing groups and hopefully you’ll have luck there too. In the UK, I live close to London so can travel to the BCS Sigist meetings quite easily, where I get to meet a lot of testers and compare notes.
I saw the concept of an online database recently discussed in the comments thread on James Bach’s blog, so you might want to read James’ response to Adam Goucher.
But who would define what “contributed sufficiently to the test community” meant? - probably most of the people involved in setting up the certification schemes could probably qualify in those terms.
I’ve certainly felt a need for a sense of belonging before, and I think people achieve that in different ways. For me, reading books like “How to Break Software” and “Lessons Learned in Software Testing” made it obvious to me that other people thought about testing in similar ways to me and I could learn a lot from them. I also have a fondness for Beizer’s “Software Testing Techniques” which some people think follows a different mindset, but I still include it in my list of favourite software testing books.
Some people require a ‘membership badge’ or ‘certificate’ to belong, while others can feel an affinity towards someone via their writing on blogs or in magazines like Better Software or Software Test and Performance Magazine.
I encourage you to build your own community around you by cordially engaging with all the testers you meet, including those that hold views you perceive as opposed to yours.
And I encourage you to go to some conferences, I gained a lot from conferences. Certainly you can gauge, in highly subjective relative terms, how you compare to other testers by meeting a lot of them in a single venue. You will undoubtedly meet people who you would associate as belonging to the same ‘community’ as you. I have a circle of testers that I seek advice from via email and phone calls - most of whom I met at conferences and software testing groups.
Welcome to Software Testing,
Alan
Excellent!
I went for the same experience. My conclusions are that contents of these courses are made by people who probably were involved in testing 20 years ago. The contents are out of date and the examination is made by people with no understanding in evaluation.
I argue more than once with the instructor about the irrelevance of the topics and the lack of realism about the real facts of our activity. The answer was always the same…”just learn exactly what it is in the handbook, because the test’s question is based on it…forget about what you think!”.
I needed the certification to feed the ignorance of the contracting agencies. I feel like an apple that as been grown with chelmicals but I was given a label saying that I am organic.
We should make a differentiation between the idea of providing testing practitioners with some sort of qualification, and the courses that currently provides these qualifications.
I strongly believe that those today provide certification do not contribute to the standardization or improvement of the practice. They are there not because of the validity of what they do, but because they were the first in the market place.
In sum; this is a great article. Glad to find in this activity people with their own mind. People who dare to say that the kind is naked, while most are afraid of do so.
Hi Rene,
Thanks for the comment.
I think the exam puts the trainers in a difficult position. If the trainees have come to learn how to pass the exam then the trainers do a fine job. If, as it sounds you wanted, the trainees want to learn more about testing then the trainers have to keep coming back to the syllabus and the rote answers because the exam values that above ‘real world’.
One step towards differentiating between a qualification and courses that lead to the qualification would require testers to study for the exam themselves.
I encourage every tester - if you do place a value on the certification, then study for it on your own. Learn some exam techniques. Learn the syllabus. Pass the exam - or fail the exam, learn from that, then sit the exam again - you will still save money instead of going on a course.
Your statement about “first in the market place”, rings all too true. Hence the ‘need’ for a ‘global’ standard certification like the ISTQB. I could cynically view the ‘need’ in terms of ‘we need to compete against any local certification’ but then that kind of monopoly ‘is’ good because then we all speak the one language. Hooray for Esperanto.
Thank you again for commenting,
Alan
I found this a very thought-provoking post, and enjoyed reading the slides and all the comments.
This is particularly true since I am deep in the throes of writing ISTQB Advanced level questions. My main reason for doing this was because I found the current questions to be absolutely useless (I could say cr*p…) - if a group of very involved, up-to-date, hands-on, highly experienced Test Managers could so easily fail the Test Managers’ Advanced exam, it was pretty obvious that the exam had no relationship with reality. (I will be up-front and admit that it doesn’t do any harm that someone is actually paying me and my team to write better questions…).
So, I have to say that I am ambiguous about certification. The negative points have mainly been written already, so below is a list of my “almost-pros”.
I’ll discuss here ISTQB as I don’t have any experience with ISEB and the way that works.
1. As Walter (and Tobbe, paraphrasing Dorothy Graham) wrote - compare it with any other type of training. The certificate (foundation at least) just shows that you are not completely ignorant in the field. After that, it’s up to the interviewer to check.
2. As Sarah wrote - the testing profession is still banging up against the brick wall of those who don’t see it as a profession. Witness the types of job ads even now-adays asking for “excellent students with 85 grade average, for work as testers” !!!!! If the fact that there are certification programs out there can help even a tiny bit to counter this, I’m for them.
3. The Advanced exams, at least the ones we are creating today, require that the examinee be able to analyze real-life scenarios. They take into account the fact that people taking these exams need to have many years’ experience in real life testing. Unfortunately they are still multiple choice, but that just means that they are very difficult to write ;-)
4. The commericalization of the training. Firstly, the ISTQB foundation certificate doesn’t need a course. Testers should be encouranged to really learn, and experience (if possible) the profession of testing, if they feel they must do the exam. True, the exam still costs money, but I don’t think that’s a bad thing, as it separates the serious testers from those who are doing testing on their way to being developers (perish the thought!!). Secondly, we are beginning to see signs that the managers of testing teams who are ordering Advanced training are becoming involved in the content of the training. They are not willing to pay for training that doesn’t relate to their needs. They also want to assess the trainer before agreeing to have him/her teach their staff. I think these are welcome changes.
5. Danny asked whether you have to be reasonably happy with the syllabus and certification in general in order to help change it? Possibly. At the very least, you have to be passionate in not liking the current situation, as effecting change is an uphill battle.
6. And last but not least. I think there are positive signs of change in ISTQB. As an example - the old advanced syllabus (2003) was really horrendous. The new one (from 2007), although of course not perfect, is light years better. Maybe there’s still hope?
In short, it’s easy to complain about certification. It’s another matter altogether to try and improve it. Well done to those who try, and I hope that we’re not being too naive in believing we can actually make a dent!
Hi Debi,
I wish you well in your endeavor to make a dent in the certification. I have my own suspicions about who will come out with more dents - you or the certification? I suspect you might come out worse.
A few comments on your comment though. I think that some of your statements don’t help the cause of certification, in the way you would like:
- I would not want to stop “85 grade average” students starting their career as testers. I started my career as a ‘junior’ tester and worked my way up. I hope certification schemes do not stop anybody working as a tester who wants to work as one. I think I started my career as a tester by answering a job ad much like the one you quote. As a junior tester I was mentored by more experienced testers. Essentially serving an apprenticeship.
- I could easily class myself as a serious tester and I continually work to improve my development skills. I do not believe that a focus on testing should preclude that person from development. I also believe that serious developers exist who have serious interests in testing and they want to become better at both developing and testing - I don’t think certification helps them.
- I found multiple choice questions which require the examinee to analyse real-life scenarios, not only hard to write, but also hard to answer since I could easily view more than one answer as credible, but had no space to state my reasoning for that, in the end I would guess and try and mind-read the examiner.
- I fully support people tailoring the content of the training they buy to meet their needs. They don’t need certification to do that though.
Thanks for leaving the comment. And best of luck with your denting.
Sorry it’s taken me so long to come back and look at your reply - somehow, life got in the way!
Of course I agree with your first 2 comments - I am sure there are many serious developers who have a serious interest in improving both their developing and testing skills (I hope that I was one of them, since I was the “weirdo” who made the move from development to testing…), and I would love to see more junior testers being mentored by senior testers with a view to making testing their profession.
Maybe in the UK, these things exist to a certain extent. But unfortunately, here in Israel, too many testers have different experiences.
The “85 grade average” students may one day become wonderful testers, but as long as the majority of the (non-testing) industry here still looks at testing PURELY as a stepping stone to development, and the job descriptions are such that you can tell that almost no-one can be bothered to write something intelligent about testing, those of us “who believe” have to try using as many tactics as possible…
I have to agree with you also about answering real-life scenario multiple choice questions - the first set of examinees who got my exam all failed!
Anyway, it takes me a long time till I stop bashing my head against brick walls, so thanks for your good wishes, and keep your fingers crossed :-)