Why I do not like to be a tester in Agile project
Lucjan StappWarsaw University of Technology
Stowarzyszenie Jakości Systemów [email protected]
Lviv November 7th 2013 2/37
Scientific worker at Warsaw University of TechnologyISTQB® CTAL – TM, ISTQB® CTAL - TA
Author of more than 40 papers, more then 10 are connected with testing;
Acting vice-president of Stowarzyszenia Jakości Systemów Informatycznych (Polish Testing Board);
Member of ISTQB® Dictionary Working Group;Member of ISTQB® Exam Working Group;
I believe(d) in Agile methodologies.
Lucjan Stapp
Lviv November 7th 2013 3/37
Agile manifesto
In February 2001, 17 software developers met at the Snowbird, Utah resort, to discuss lightweight development methods.
They published the Agile Manifesto (Manifesto for Agile Software Development) to define the approach now known as agile software development.
Lviv November 7th 2013 4/37
Agile manifesto
We are uncovering better ways of developing software by doing it and helping others do it.Through this work we have come to value:
That is, while there is value in the items on the right, we value the items on the left more.
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiationResponding to change over following a plan
agilemanifesto.org
Lviv November 7th 2013 5/37
Agile manifesto
As a result of each iteration we obtain the working software
Only iterativeincremental
approach
Lviv November 7th 2013 6/37
Basic Agile principles:
Working software is the primary measure of progress
Informal (face-to-face) communication,
Self-organization and motivation, Inspect and adapt, the team reflects on how to become more effective, then tunes and adjusts
its behavior accordingly
Customer collaboration: welcome changing requirements, even late in development.
Improving the chances of understanding what the customer wants
Agile manifesto
Lviv November 7th 2013 7/37
Agile manifesto
Agile methodologies (between others): eXtreme Programming XP, Scrum, Kanban, Dynamic Systems Development Method, Adaptive Software Development, …...
eXtreme Programming XP
Lviv November 7th 2013 8/37
How it works?
Agile teams are more effective in desired functionality then “traditional” teams.But less in quality.However testers are responsible for quality. What can we do to change this aspect, to obtain higher quality in Agile teams?
Results of Dr. Dobbs Journal Investigations(2008)
Quality Functionality ROI (money) Schedule0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.0
0.3
0.1 0.1
0.2
0.2
0.2
0.1
0.60.6
0.3
0.4
0.5
0.6
0.4
0.4
Main aspects
Ad-hoc Waterfall Classical Iterative Agile
Lviv November 7th 2013 9/37
Higher ROI.
Agile teams work in more effective way (but not harder) and should propose needed functionality earlier, hence there is a shorter path to the market and greater profit.FaithDDJ found also that people believe that Agile teams product higher quality (however it is NOT true) then traditional teams, hence one can observe higher satisfaction of stakeholders.
How improve quality?
Lviv November 7th 2013 10/37
Power of Three. “Whole Team Approach”
Whole Team usage means involving all of the people with the knowledge and skills necessary to ensure project success.
quality is the responsibility of the Whole Team Testers work closely with both developers and the business
representatives to ensure quality across all test levels
How improve quality?
Lviv November 7th 2013 11/37
Power of Three. From the individual agile tester perspective “Whole Team
approach” means daily collaboration with both developers and business representatives.
Testers should transfer testing knowledge to non-testing team members and
influence the development of the product. ensure the right level of integration with developers and the right
level of collaboration with customers. How to do it?:
the daily stand-up meeting, involving all members of the team, at which project status is discussed and any impediments to progress are highlighted.
How improve quality?
Lviv November 7th 2013 12/37
Daily scrum
Scrum
Lviv November 7th 2013 13/37
Teams
Teamwork is one of the fundamental principles in agile development.
It is important to select the right people that work well together to successfully create the desired product
Lviv November 7th 2013 14/37
Team
Galleys vs. Viking boats
Propelled mostly by oars
Lviv November 7th 2013 15/37
Team
Viking expansion from 8th till 11th centuries
Lviv November 7th 2013 16/37
Team
But
• Viking expansion from late 8th till 12th century
• Galleys are used from the 9th century BC until development of advanced sailing warships in the 16th century
• Trade and transport used galleys, not Viking knörr
Lviv November 7th 2013 17/37
Team complementThere is no division on developers and testers. Testers should be build in team.
But:
Team
• How many team members do concentrate on quality problems? (1/10, 2/10 ???);
• “Group thinking” – typically only positive;• Not enough “business knowledge” (power of three?)
• Problems with product owner.
Lviv November 7th 2013 18/37
User Story the basis of requirements Elements of User Story
As… (concrete type of user) I want… (problems to be solved) because… (desired results).
Definition of user satisfaction criteria Typically as set of acceptance tests (Done)
Test priority and story criticality can be based on the users category.
User Story
Lviv November 7th 2013 19/37
A good user story is: Independent Negotiable Valuable Estimable Sized Appropriately Testable
INVEST
User story
Lviv November 7th 2013 20/37
Testing in Agile Projects
INVEST –Testable
• Testing in Agile should be iterative • Testers in Agile must work without complete
documentation• Testers in Agile should be flexible.
Lviv November 7th 2013 21/37
User Story
Agile team does not expect very specific complicated conditions.World wide transaction system for an international bank
A fish trade company in Japan makes a payment to a vendor on Iceland. It should have been a payment in Icelandic Kronur, but it was done in Yen instead. The error is discovered after 9 days and the payment is revised and corrected, however, the interest calculation (value dating)…
From a talk by Hans Buwalda
INVEST –Testable• Tests concentrate on “simple” problems
Lviv November 7th 2013 22/37
Basic testers activities in Scrum
Testing in Agile
Beginning of the project
Release planning
Sprint planning
Understanding of the project principles
Stories estimation; questions: „What happens, if…?”
Validation of satisfaction conditions , adding new ones
Each sprint
Creation and testing in threes:Developer, business and tester
• Write and execute tests for each story,
• Write and execute functional tests,
• Automate test,• Make exploration tests.
Lviv November 7th 2013 23/37
Testing in Agile
• Write and execute tests for each story,• Write and execute functional tests,• Automate test,• Make exploration tests.
Agile Tester should be able to get quickly acquainted with the product under test get familiar with the products domain to understand product
requirements have sufficient product and/or customer related domain knowledge
be able to distinguish between critical and less critical requirements
be able to design appropriate test cases be able to evaluate test outcomes
Lviv November 7th 2013 24/37
Testing in Agile
• Write and execute tests for each story,• Write and execute functional tests,• Automate test,• Make exploration tests.
Not only Test Driven Development (MUST)
But alsoAcceptance Test Driven Development
Behaviour Driven Development Inverse test pyramid
Lviv November 7th 2013 25/37
Testing in Agile
LIkelIhood
Impact
II
IV
I
IIIL
H
M
L HM
Must Test
Should Test
Could Test
“Won’t Test”
focus of unit testing
focus of acceptancetesting
• Write and execute tests for each story,• Write and execute functional tests,• Automate test,• Make exploration tests.
Tester should be able to distinguish between critical and less critical requirements.
Lviv November 7th 2013 26/37
Testing in Agile
• Write and execute tests for each story,• Write and execute functional tests,• Automate test,• Make exploration tests.
• Continuous feedback is obtainable and can be achieved by creating and maintaining an appropriate set of automated tests.
• A tester in an Agile team should understand basics of test automation to be able to execute automated tests and to (on his own or with help of the programmers in his team) automate test cases
Lviv November 7th 2013 27/37
Testing in Agile
• Write and execute tests for each story,• Write and execute functional tests,• Automate test,• Make exploration tests.
It is not possible to instantly automate each test case. Automated testing has to be supplemented by techniques for
quick manual testing. Exploratory Testing is a technique accomplishing that.
Lviv November 7th 2013 28/37
Testing in Agile
What it means for Agile testers?
“fanatic”, „maniac”, „zealot” continuous learning
by experience ? when?
Fail fast Learn quickly Fail often Learn continually Fail cheap Learn inexpensively *
tools, tools, tools exploratory testing
*d’Amico V. The state of Agile
Lviv November 7th 2013 29/37
But ……
Module tests (also TDD) are limited in finding bugs:Capers Jones1 found that average effectiveness for unit test is between 25
- 30%.And Rex Black2 (for American market ) found that good system tests done
by independent test team achieves 85% effectiveness in founding bugs.
1Capers Jones: MEASURING DEFECT POTENTIALS AND DEFECT REMOVAL EFFICIENCY http://www.rbcs-us.com/images/documents/Measuring-Defect-Potentials-and-Defect-Removal-Efficiency.pdf
2 http://www.rbcs-us.com/images/documents/
Problems…
Lviv November 7th 2013 30/37
Solution:Two levels of testing:• Internal testing – build in Agile team• External testing – independent test team
• ATDD facilitates the use of external testing teams to perform functional testing
Typical solution
Lviv November 7th 2013 31/37
Typical solution
Release + changes
Independent test team
Release no. k
Release no. k+1
New
st
ori
es
(changes
+
defe
cts)
Release + changes
Internal tests
External tests• Acceptance
• UAT• Exploratory• Nonfunctional• Scenario based
New
st
ori
es
(changes
+
defe
cts)
Lviv November 7th 2013 32/37
But:
It means at least 3 - 4 weeks delay between founding the failure and the new version ready for re - tests.
Typical solution
Lviv November 7th 2013 33/37
AGILE == SUCCESS??
Examples
Lviv November 7th 2013 34/37
• Only f2f communication• No written requirements
• Based on old existing systems
• Product owner• Concentrate on specific
financial modules• No business representative in
team• Tests
• Mostly user acceptance• No written information about
incidents
Examples
RESULT:• 6 month delay• Business problems
•No working application
Example no.1
Lviv November 7th 2013 35/37
• Only f2f communication• No written requirements• Frequent changes• Without info to test team
• Product owner• Less of business knowledge• Concentrate on security
problems
• Tests• Long time before bugs closure
Examples
RESULT:• disaster
Example no. 2 NEW FUNCTIONALITY in bank system
Lviv November 7th 2013 36/37
• 90% written requirements
• Product owner• Concentrate on performance
problems
• Tests• 2 performance testers in team• 60% time – performance tests
Examples
RESULT:• Ready June 1st 2012• Success??
Example no. 3 Telecom application connected with EURO 2012
Lviv November 7th 2013 37/37
Thanks for your attention
Top Related