Graham Chapman
"It's possible to find defect, but impossible to find 100% correct functionality..."
Using the term "eXtremal Testing" and a number of books I'm trying to promote the idea of "tester-centered" Offshore Software Development. My goal is to show you the key pitfalls in Development Projects, related to testing and verification of Software System, and to help you to avoid such pitfalls. The article is intended for Teams and Clients of Development Projects - two always-fighting counterparts.
History Of EXtremal Programming, Brief Overview
History of eXtremal Programming (XP) is rather short. XP was invented in the 1990s as a new software development methodology. Kent Beck, a project manager in Daimler-Chrysler, first used it in one of its Development Projects. In fact, XP inherits major concepts of Agile Manifesto, published earlier. In fact, XP is the simplification of classic concepts, created for lazy-bones.
In comparison with more documentation-centric software project management processes, like Rational Unified Process (RUP), Microsoft Solutions Framework (MSF) and IEEE set of standards (SWEBOK), XP is customer and programmer centered. I mean that documentation in XP plays less important role than knowledge, possessed by individuals (Coding Group members) [myers04].
eXtremal Testing (XT) is the approach to Software System verification in front of system requirements. eXtremal Testing combines classic methods (system testing, performance testing, stress testing, volume testing, usability testing) and Agile best practices of project management [beck03].
Pitfalls In Software System Testing
In Development Projects Coding Group is not communicating with Client personally and requirements are specified on paper (sometimes with low quality). Using XP/Agile best practices is rather difficult. Cost of quality (cost of Bug removal) is rather high and may impact scope creep.
I see a number of pitfalls in Verification And Validation, when fixed-cost or time&material contract is outsourced to a remote Coding Group:
- "Software System quality mostly depends on Programmers' attention and skills" - this is wrong.
- "Tester can't understand the subject area and all Software System features" - wrong.
- "Client is the best Tester" - wrong.
Five Steps Of Software System Verification
I'm sure you know that cost of quality (defined by ASQ as cost of defect removal) increases with Software System development through phases: requirements, architecture, design, coding, testing, deployment, maintenance. I want to define five major steps in Software System verification, which should be done by Coding Group on each iteration, in each phase, for each new build:
- Step 1. Compilation,
- Step 2. Unit-Testing,
- Step 3. Code Peer Review,
- Step 4. System Testing,
- Step 5. User Acceptance Testing (UAT).
Compilation is done by Programmers and integrators by means of software compilers. Existing tools (Microsoft Visual Studio, Eclipse, IBM Software Architect and others) are very powerful and help to reveal major code syntax Bugs.
Unit-Testing is a done by Programmers and unit-tests (testability elements). Test-Driven Development (TDD) explains the approach to unit-testing [beck03].
Code Peer Review is a method of Bug revealing without Software System running. Peer Review is done by Programmers from another IT Project [wiegers01].
System Testing (including manual testing and automated testing) is done by Testers and is reported through Bug tracking system. Also called as alpha-testing, this step is performed before showing the Software System to Client [kaner99].
User Acceptance Testing (UAT) is performed by Client and is known as beta-testing. Each Bug reported on this step may cost you money. Your major goal in any IT Project is to minimize the amount of Bugs found at this step by Client [kaner02].
Fire Your Programmers And Hire Testers
I know that some of the sentences below are provocative. I know that you love your Programmers and you respect their work. I know that you feel that they are the people who create the Software System. This is not true. Start with the small changes:
- Limit Your Programmers in Time
- Motivate Testers to Find Bugs
- Create Your Coding Manual
Limit Your Programmers in Time. If they work until the result is achieved - they will work a lot, they will do their best, they will re-factor code, they will do add extra functionality, they will fill themselves happy and proud of the result. Your IT Project will suffer. Limit them in time and ask to implement requirememts as fast as they can - then give them another task. Project manager, don't let your Programmers to manager your project! Stop them as soon as functionality is implemented - no matter how many Bugs the code contains.
Motivate Testers to Find Bugs. Give $5.00 for each new Bug found by your Testers and tell them, that if Software System doesn't have Bugs - they will be fired. Software System without Bugs means that your Verification And Validation team is wasting your money. Motivate them to crash the Software System.
Create Your Coding Manual. Create a 10-page document with instructions on how to name variables, how to comment code, how to create cycles, loops, functions, build arrays, etc. Work with this document, not with your "professional" Programmers. The time you spend with your Programmer is wasted. The time you spend for Coding Manual is your asset. Make sure you award the best Programmer - the person who is the most compliant to your Coding Manual.
First writen online on 6/17/2006
