Managing Quality Through the Entire Development Life Cycle

Facilitated by Tim Bishop

Tim will share his learnings and practices drawn from his rich experience in testing and supporting nonprofit software products.

Notes

These are notes from Leo's session on quality

QA

• is an insurance
  • creates stability and quality
• can't be an afterthought
  • needs to be part of the design and development process
• Example
  • no new staffing at Empower, went form 1800 to 300 bugs
• Example:
  • Went from 3 month cycle release to 1 month cycle
• saves many hours
• Time in testing or time in technical support

QA Team

• Core Based
• Customer
• Sourceforge

Tools/Techniques

• Contiuous Integration Server !Important
  • Automatic build/compiling testing
  • Automated Test is run with Empty DB in Memory
  • Unit Test
    • Smallest part
• Don't commit without running tests
• Test Driven Development
  • Define Tests first
  • Fail first, then develop that pass tests
• Code Coverage !important
  • 80% must be solid (20% risk)
    • Drupal does 70%
• Types/Levels Testing
  • Unit Testing
    • Example: Testing methods of objects
  • Functional Testing
    • Often bugs are at this level.
  • Integration Testing
  • Different Layers
  • Regression Testing
• Web Testing
  • Start with blank DB
  • TestNG (Java Tool)
    • Can drive web-sites
• All Automated Tests have to Pass before QA looks at it
  • Often new features, no test written
  • Simple Manual Tests First
    • Can ask a handful of people to run these simple tests
• Bahvioir Testing
  • What should a method do (not the possibilities)
  • Instead of Unit Test
• MOCK
  • Mock Objects
• Cruise Control
  • Sends info to IRC
• Hudson
• Ship Product with Test
  • User can run Tests
  • Email results of tests
• No Test, No Code
  • If you can't write Unit Test, then don't write code
• Weekly Code Review
  • Look at Unit Tests
• Test Harness
  • Collection of Unit Tests
• To Start Testing
  • Look at Bugs
    • Write a failing tests for the bug to recreate bug
• 3rd Party Integration
  • Write Tests for their API
  • Customers are not concerned with who broke what
• Start Testing from Base of Stack
  • DB, Domain Object, Web Interface
• Offer Testing in Contract
  • Support versus Testing hours
  • Offer no testing and $500/mo support
  • Offer testing and $100/mo support
• In open source community, its important to push for testing

Issues/Debates

• Getting QA started in organization
  • How to write a Unit Testing
  • Use Pre/Post Conditions
  • Use of testing tools
  • Creating value for QA (especially with development/engineering)
    • More valuable that engineering
• A lot of Overhead
• Tech Support is not QA

Agile

• QA works well with Agile-like process

Benefits

• Can find exact issues easier
  • Saves many hours of time when looking for problems

Project Size

• Small
  • Unit Testing
  • Don't commit without passing
  • Regression Testing
  • Build in Test Harness
  • OFten smaller projects are used in bigger projects
  • Automating Project Setup, Common Tasks

Overall Quality

• More than just testing
• Tests are great, but there still needs to be in-person/manual quality checking