Trend Quality Assurance
Edward TsaiEdward Tsai 蔡木本蔡木本趨勢科技全球軟體研發部協理
April 22, 2003
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineContents
Software quality assurance overview Testing skill and debug tools
Testing concepts. Testing methodology. Testing tools.
The importance of QA Automation Software testing in enterprise
environment
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
Overview
What is the quality? What is QA role in Trend? How many different type of testing
strategy? What is the typical software life-cycle? What is the QA challenge and future? What is Trend QA organization?
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
What is the quality?
2 level of quality: Level 1: product defect rate and reliability. Level 2: product quality, process quality,
customer satisfaction.
Quality from professional views: Conformance to requirement. Fitness for use.
For example: Cadillac and Chevrolet car, Which has good
quality?
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineWhat is QA role in Trend?
The purpose of QA engineers: To discover all possible errors in
development cycle. To discover every conceivable fault or
weakness in product. To simulate customer’s focus and need. To measure the quality level of product. To delivers the quality report to service
team.
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineHow many different type
testing strategy? Black-Box testing:
Derived from the functional design spec, without touching the internal program structure.
White-box (or Glass-box) testing: Require knowledge of the internal program
structure and derived from the internal design spec or code.
Trendmicro: Black-box testing and will migrate to Gray-box
testing.
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
What is the typical software life-cycle?
Concept. Requirement. Design. Implementation or coding. Test. Operation and maintenance.
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineDOT-U model
1
2
3
4
5
6
7
8
9 10
11
Requirements
specification
Requirements
verification
Functional design
specification
Functional Design
verification
Internal design
specification
Internal design
verification
Products simulation
Usability test
Code and spec
modification
System and
Acceptance
validation
Function validation
Code and spec
modification
Code and spec
modification
Code and spec
modification
Code
Code verification Unit validation Integration validation
1
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
What is Trendmicro software life-cycle?
Requirement. High-Level design/Planning. Detailed design. Construction. Pre-release. Closure. Service.
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineWhat is QA
challenge/future?
Professional discipline requiring trained, skilled people.
Advances in technology/tools to create complex software in GUI, Client-Server, Intranet/Internet architecture.
Multi-platform, multi-language. Many/complex compatibility software
testing. Keep on process improvement. Test automation is not mature yet.
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
Defect Distribution
Requirements56%
Design 27%
Code 7%
Others 10%
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
The cost of defect
Cost
Time
Alpha Beta Release
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineTester-to-Developer Ratios
The lesson: The importance of testing is now recognized. Vendors are willing to pay for proper testing.
A dramatic shift: Historically, for Mainframes : 1: 5-10. At Microsoft, 1992 : 2:3. At Microsoft, 1995 : 1:1. At Microsoft, 2000 : 2:1. At Microsoft, 2005 : ? TrendMicro, 2000 : 1 : 2 TrendMicro, 2003 : 1 : 1
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineWhat is the Trend QA organization?
Project - LeadEngineer Q AEngineer Q AEngineer Q A
Project AM anager - Q A
Project - LeadEngineer - Q AEngineer - Q AEngineer - Q A
C ertifcationM anager-Q A
Project - LeadEngineer - SWEngineer - SWEngineer - SW
Test developm entM anager-Q A
Lab engineerLab engineerLab engineerLab engineer
Lab adm inM anager-Q A
Project - LeadEngineer Q AEngineer Q AEngineer Q A
Project BM anager - Q A
D irector-Q A
G lobal D irector-Q A
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineTesting skill and debug
tools
Testing methodology.
Test case design.
Testing tools.
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
Testing methodology
Verification testing: Reference to Dotted-U Model. Verification methods : Inspections, walkthroughs, Technical
reviews, others. The verification tool : Checklists. Include: Requirement, Architecture, Plans, Design Spec,
code, Documentation. Validation testing:
A process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirement.
Include: Unit testing, integration testing, Usability testing, Function testing, Regression testing, System testing, Acceptance testing (Alpha, Beta testing).
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
Testing case design
Go through all testing synopsis at every checking point. Task-Oriented Functional Test (TOFT) Forced-Error Test (FET) Boundary test Volume Test Stress Test Performance Test Security Test. Configuration Compatibility Test Documentation Test Trend Standards in Product
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineTest Coverage
Test coverage has three components: Requirements coverage. Function coverage. Logic coverage.
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
Example
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
Example
Input a Directory name to install a product in NT system. Task-Oriented Functional Test (TOFT)
• Input a legal directory name with length between 1 to 255 character. Forced-Error Test (FET)
• Read privilege only directory, Non-existed drive, more than 255 characters, Not enough disk space.
Boundary test• Input a space or null character, Input one character, Input 255
characters, Input 266 characters. Memory leak after install complete. Volume Test
• Install it with 30 MB free disk space. Stress Test
• Install it with CPU utilization more than 70%, Free visual Memory left less than 1 MB, ...
Performance Test• Memory usage, CPU utilization, Any network traffic generated,...
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
Example
Security Test.• Does it can be installed from every users or Administrator only.
Configuration Compatibility Test• N/A
Documentation Test• The description are properly and correctly
Trend Standards in Product• Does it install under c:\programs files\TrendMicro\PC-cillin 98?
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
Testing tools
Testing documents database: Test manager, TCMT.
Bugs tracker system: PVCS Tracker 7.0.
Automation tools: Visual test 4.0, Robot J, ATP. Winrunner.
Source code Coverage: PureCoverage.
System tools: Loadrunner, Load test, Winbench.
Debug tools: Boundchecker, Quantity, Purify, SoftIce.
RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line
Q & A
Top Related