A Case Study in Test Management
Tauhida Parveen Scott Tilley George Gonzalez
Dept. of Computer Sciences Dept. of Computer Sciences Software Quality Management
Florida Institute of Technology Florida Institute of Technology Sabre Holdings Inc.
tparveen fit.edu stilley@c fit.edu @ s. george.gonzalez@sabre.com

ABSTRACT
Testing is an essential but often under-utilized area of software
engineering. A variety of software testing techniques have been
developed to effectively identify bugs in source code, yet these
techniques are not always fully employed in practice. There are
numerous reasons for this, including the difficulty in mastering
the complexity of managing all of the test cases for large-scale
projects. Test case management involves organizing testing
artifacts (e.g., requirements traceability data, test cases, and
expected results) in a systematic manner. To be successful, test
case management requires a high degree of discipline to
accommodate the large volume of artifacts under consideration.
This paper presents the results of a case study in centralizing test
artifacts in an industrial setting to aid better test management.
Several of the challenges in adopting this approach are discussed.
In response to these challenges, recommendations on how to
better leverage test case management are offered.
Keywords
Software testing, test management
1. INTRODUCTION
Software testing is a process to identify the correctness,
completeness, and quality of developed computer software. It is
an integral part of software engineering. It encompasses the
concepts of demonstrating the validity of software at each stage of
the development life cycle and the authenticity of the final system
with respect to the customer’s requirements [1]. However, in
application development projects, testing is often not given
enough resources, time, and priority until initial development is
completed. With competitive pressure and the increasing cost of
downtime, organizations have started to introduce testing at
earlier stages of software development while others are striving to
find effective testing strategies. Few organizations have
established the basis to measure the effectiveness of testing.
Without testing standards and a proper test management strategy,
the effectiveness of testing can not be measured or improved.
This paper focuses on the test management part of the testing
process.
Test management is a method of organizing test assets and
artifacts such as test requirements, test cases, and test results to
enable accessibility and reuse. This paper compares and contrasts
the theory of test case management against what commonly
happens in industry with respect to manual test cases. It describes
a case study of the authors experience in applying academic best
practices for software test case management in industry and the
outcome of such best practices. The paper also outlines an
approach to streamlining test case management and how to
enhance productivity through test case management.
The paper is organized as follows: Section 2 gives an overview of
testing and the common techniques that are followed in the testing
process. Section 3 describes a case study of the experiences of the
authors in implementing a test management strategy. Section 4
illustrates the impacts of establishing test management, the
challenges faced and some recommendations to aid in efficient
test management process. Section 5 summarizes the paper with
some direction for future research in this area.
2. SOFTWARE TESTING
Research shows that software quality consumes more than fifty
percent of the cost of the software and this number is even higher
for safety critical software [2]. One way to support quality
assurance is efficient software testing. Testing refers to the
execution of software with an input (known as the test case) and
observing the result of the input. The output data helps in
understanding the behavior of the system and provides data to
ensure the conformance to specification of the software.
The goal of testing is to efficiently identify any high impact
deviations from expected results, so that these deviations can be
corrected or prevented before the software is released. The value
of testing also encompasses goals such as the reduction of the
number of defects and increase development productivity through
continual feedback.
2.1 Testing techniques Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage, and that
copies bear this notice and the full citation on the first page. To copy
otherwise, to republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
ACMSE 2007, March 23-24, 2007, Winston-Salem, North Carolina, USA
©Copyright 2007 ACM 978-1-59593-629-5/07/0003...$5.00
Software testing is a collection of techniques used in the process
of verifying that a piece of software is free from technical errors.
Three well-known testing techniques that are commonly used in
the industry are black box, white box, and gray box testing [6].
Black box testing treats the system as if it were a “black box.”
This testing technique requires no knowledge of the internal code
of the system during the testing process and with little regard in