Skip to main content

[More Conferences]

Test Focus Group - London - 21st March 2017

I hosted the QA in Devops Session. Three different discussion sessions over the day. Read more for notes and details.

Test Focus Group - London - 21st March 2017 Thumb Image

Test Focus Groups - London March 2017

At the Test Focus Groups] March 2017 held near Heathrow, London, I hosted the three Devops and QA sessions.

The sessions were recorded so hopefully I will be able to link to more material in the future. I didn’t make many notes because the discussions were pretty fluid and flowed well.


I hosted a “Devops and QA” session. We had a mix of participants ranging from people who were actively working on DevOps and people who were interested.

We learned a lot about other people’s working environments, some of which seemed actively hostile to a DevOps Approach.

Basic points that I remember:

  • QA in DevOps does mean Quality Assurance rather than testing, so if only testers are doing QA then it really isn’t DevOps
  • Agile, DevOps are related but you can do one without the other
  • DevOps still needs testing and still needs exploratory testing
  • Yes DevOps tries to automate as much as it can
  • Change architecture for effective DevOps
  • worth reading “The DevOps Handbook
  • people recommend reading “The Phoenix Project”
  • Managers might find The DevOps Adoption Playbook a good place to start


Prior to the event, I made some notes to prompt discussion and my memory if necessary. I include them below for your perusal and delectation.

  • What is Agile?
    • merge testing, programming, designing, etc. into “Development”
  • What is DevOps?
    • merge “Development” and “Ops”
  • How is DevOps different from Agile?
    • releases, infrastructure as code, monitoring

Testing in Devops

  • Does DevOps change the test process?
  • Does Testing Fit into DevOps?
  • Do Testers Fit into Devops?
  • Do Test Managers Fit into Devops?

What Practices Support Devops

  • Continuous Integration
  • Continous Delivery
  • Automated Application Execution and Assertions - beyond Unit testing

What happens when you ‘do’ DevOps?

  • Architecture changes:
    • smaller systems - micro-services, web services
    • health checks
    • APIs
    • easier to automate
    • easier to ’test’ in smaller chunks
    • loose coupled

Possible Topics

  • What is DevOps?

  • Experiences of DevOps

  • Testing in Devops

  • Role Changes

  • Roll back, path forward - Idempotently

  • Configuration as Code - jenkins, Chef, Puppet

  • Containers - share os, isolate memory, I/O etc.

  • Infrastructure as code - Containers Technology - vagrant, docker, Kubernetes,

  • Monitoring - Graphite, Kibana (Elastic Search. Logstash)

  • “Shift Left”

  • Test Data - discovery, masking, creation, ‘golden deploy’

  • “serverless” - always in production environments

  • “Anti-Fragile”, Reliable - chaos monkey

  • Assurance - build the right thing

  • APIs

  • Cloud - risks, benefits, issues

  • Leverage Platform, Infrastructure as a service

  • 3rd part applications impact on devops

  • Lean - Make work visible, Limit WIP, Reduce Batch size, Reduce Handoffs, Identify & Remove constrains, eliminate waste

  • Fast Feedback - How

  • Learning Culture

  • Poor adoption

    • Devops project
    • ownership
    • silos
    • agile didn’t work - try devops
    • replace agile with devops
    • remove ops

Performance Testing as Code - e.g. Gatling, Ruby JMeter

Where does it fall down?

  • Risk Analysis
    • no ’tester’ to consider risk and explore for risk
    • overconfident safety net because of the ‘amount’ of automated execution - #tests, length of test run, code coverage
  • focus on ‘speed’
    • speed might not be more ‘important’ than ‘assurance’
  • backups
  • roll backs
  • checks during automated deployment
  • monitoring not monitored


  • Finance Market Org
    • move to ‘devops’ after Agile
    • sub team within dev team responsible for Ops support - dev and ops
    • pragmatic rather than ‘idealised’
  • Payments
    • multiple teams - different approaches
    • teams support app in prod, ops support environments
    • smaller teams with smaller systems have more control
  • Others
    • Small outsourced dev team - the only team (ops + dev) but none of the controls
    • “on call”

Social Media

[List of Conference Talks]