Abstract
Software testing is utilized to find defects in requirements, coding and design of the software or product . By software testing users tend to find how correct, complete, secure is there product and be assured about the quality of the product as compared to the laid down specifications .
The importance of testing is such that the question always arises about the strategy one should adopt for testing. This report understands and compares the most prevalent testing techniques in software testing for detecting errors.
Methodology of the research is qualitative as research articles, journals and book were read to understand and analyze the Testing techniques and it is concluded that there are various testing techniques like black box , white box and grey box technique to test a software and provide users with a defect free product which is usable and able to perform reliably along with automation which helps users to regress testing thoroughly.
Introduction :
The aim of testing is quality assurance, reliability estimation, validation and verification. Software testing plays vital role in software quality assurance and represents a review of specification, design and coding. The importance of software testing is to confirm the quality of software system by testing thoroughly the software in carefully controlled and monitored circumstances, another aim is to be confident of the completeness and correctness of the software, and finally it uncovers hidden errors. The three main techniques that are used for identifying errors are:
1) White Box Testing Technique: It is the investigation of the logic in code which runs internally. Testers require working knowledge of the programming and code.
2) Black Box Testing Technique: This form requires no knowledge of internal working of the system under test. Only the fundamental knowledge of the application is required.
3) Grey Box Testing Technique: This technique is a mix of both black and white box testing techniques, it requires the limited knowledge the code of the application and the fundamental knowledge of the application.( Software Testing Techniques in Software Development Life Cycle (Meenakshi, D., Naik, J. S., & Reddy, M. R. ,2014))
Software Testing is performed for comprehending software quality and also for improving it Thus, the goal of testing is detection of errors systematically within a required amount of time and also with a right amount of effort. Software testing is also an important component of software quality assurance (SQA), and a number of software organizations are spending up to half of their resources on testing.
Four main phases of testing are as follows: (A Comparative Study of White Box, Black Box and Grey Box Testing Techniques (Khan, M. E., & Khan, F. ,2012))
Detection: Errors are found in this phase
Prevention: This phase finds the information to decrease errors and to clarify the requirements of the system
Demonstration: This phase demonstrates the usage of the system with various acceptable risks.
Quality Improvement: By performing rigorous and systematic testing on system under test , error can be drastically reduced and quality of the software as such can be greatly increased. For systems like launching of ICBM’s( Intercontinental ballistic missiles ) testing can be much costlier as risk associated with failure are almost not acceptable. Hence test planning, test case writing and basically verifying and validating the system using prevalent testing techniques becomes even more important.
The basic purpose of software testing is verification, validation and error detection in order to find various errors and problems – and the aim of finding those problems is to get them fixed.
Literature Search
Paper 1 :
A Survey on Software Testing Techniques using Genetic Algorithm (Sharma, C., Sabharwal, S., & Sibal, R. ,2014)
Merits:
1) Automatic generation of test case and test data is proposed
2) Regression testing can have minimal number of test cases by deleting redundant test cases by applying GA
3) GA successfully generates test data from the UML state diagram
4) Increase in testing efficiency by testing critical path in CFG (control flow graph)
5) GA is used in focused software usage testing by identifying the nature and locations of errors thereby improving the quality of software and efficiency of debugging activity
Demerits :
1) The test cases are generated from the output domain. Results have shown that this approach can generate test cases from output domain with high efficiency. The experiments were conducted only on small programs
2) The effectiveness of test cases generated from the proposed fitness function is not evaluated with other test case generation techniques from the UML.
Methodology :
Mixed research ( qualitative and quantitative ), experiments are conducted based on previous studies and research to determine the efficiency of the testing techniques
Paper 2:
Software Testing Techniques and Strategies (Abhijit A. Sawant , Pranit H. Bari , & P. M. Chawan, 2013)
Merits :
1) Covers goals of software testing verification and validation, where verification is reviews and validation being executing the code, Priority coverage , balanced and traceability , deterministic.
2) Testing principles are covered like: defining test plan, design effective test cases, starting testing early, and equivalence portioning and boundary analysis and should be performed at different levels by different persons.
3) Covers testing techniques like Static testing ( manual testing) , Dynamic testing ( Automated testing)
4) Covers white box techniques like basis path, loop, and control structure
5) Black box covers equivalence portioning, Boundary value analysis, cause effect graphs, comparison, fuzz, model based techniques
6) Software testing strategies are covered like unit testing, integration testing , system and acceptance
7) Covers performance , reliability and security testing
Demerits:
1) Static testing and manual testing differences do not come up very clearly as static testing is review based while manual is more into Functional testing.
Methodology:
Qualitative as authors have researched a lot of journals and papers
Paper3:
Software Testing Techniques in Software Development Life Cycle (Meenakshi, D., Naik, J. S., & Reddy, M. R., 2014)
Merits:
1)Covers testing types like Unit testing done at code level, integration testing done at module level, function testing , system testing , acceptance done by users, beta testing done by customer
2) Software testing life cycles phases like requirement study from the users , design where componenet are identified , Construction phase where test execution and evaluation is done and deployment phase where test summary report is created.
Demerits:
1) Boundary value analysis as technique of creating test cases is not covered.
Methodology:
Qualitative as authors have researched a lot of journa
ls and papers
Paper4:
Software Testing Techniques (Ghuman, S. S. 2014)
Merits:
1)different levels of testing like Unit testing done at code level , integration done at module level, system testing done at system level, acceptance testing done by users, alpha testing , beta testing and regression testing is covered.
2) Testing techniques like Static with focus on reviews, dynamic with focus on execution of software, functional black box and structural focus on white box are covered.
3) Testing goals like Verification and validation, priority coverage, balanced, traceable and deterministic.
Demerits:
1) Does not cover Nonfunctional testing aspect like performance, reliability, security, installibility testing.
Methodology:
Qualitative where a lot of research articles and journals are referred by the author
Paper5:
A Comparative Study of White Box, Black Box and Grey Box Testing Techniques (Khan, M. E., & Khan, F. ,2012)
Merits:
1) Techniques like Black box which cover fundamental aspects but not the internal working, white box which investigates internal logic of software, grey box which is mix of both the techniques
2) White box is explained in detail with control flow testing, branch testing, basis path testing, data flow testing and loop testing
3) Black box technique with Equivalence portioning, boundary value, fuzzing, cause effect graph, all pair testing is explained
Demerits:
1) Automation testing technique has not been given focus
Methodology:
Qualitative where a lot of research articles and journals are referred by the author
Paper 6:
Foundation of software testing (Black, Rex, 2011)
Merits:
1) Black box testing covers both functional which is specification based and nonfunctional like usability, maintainability, reliability aspects of it.
2) Black box testing techniques cover Use case testing (transaction by transaction basis from start to finish), State transition testing (based on finite number of state system can be in), and decision tables like cause effect graphing.
3) White box testing is covered with all subsets.
4) Experience based testing technique like exploratory testing and Error guessing is proposed
Demerits:
1) Does not cover Generic algorithm to create automatic test data and test cases
Methodology:
Qualitative as lot of research papers were read by authors.
Paper 7:
Latest Research and Development on Software Testing Techniques and Tools (Chauhan, R. K., & Singh, I. ,2014)
Merits:
1) Different types of white box testing techniques are as follows: – Basis Path Testing Loop Testing Control Structure Testing
2) Different types of Black box testing techniques are as follows: – Equivalent Partitioning Boundary value Analysis Cause-Effect Graphing Techniques
3) Grey box: It uses internal data structures and algorithms for designing the test cases more than black box testing but much less than white box testing.
4) Automation tools like Ranorex, RFT, janova are explained which help create automated test cases which can be run easily on newer builds.
5) Latest research in testing like test driven development, iterative and incremental testing is explained
Demerits:
1) Does not cover Generic algorithm to create automatic test data and test cases
Methodology:
Qualitative as lot of research papers were read by authors
Paper8:
Comparative Study of Automated Testing Tools: Selenium, Quick Test Professional and Test complete (Kaur, H., & Gupta, D. G. ,2013)
Merits:
1) This research evaluate three major tool vendors that are Selenium , Quick Test Pro (QTP) and Test Complete on their test tool characteristics, test execution capability, test resorting capability, scripts reusability capability, play back capability
2) There are two types of test tools. Open source test tools• Commercial test tools• Open Source Test tools- These test tools are free for the users to use. It can be downloaded from the internet or can be obtained by the vendor without any charges e.g. Selenium, test tools such as QTP and TC are not free.
3) Research focuses on QTP, Selenium and test complete which are mostly used tools in the testing industry
Demerits:
1) None.
Methodology:
Qualitative as lot of research papers were read by authors
Summary for all the above mentioned papers clearly shows that papers 2 and paper 7 discuss about Black box technique with focus on equivalence partioning, boundary value analysis, cause effect graphing which are prevalent techniques for better test cases and better integration, system testing of the system, also mentioning white box testing techniques with focus on loop, branch and basis path testing for better unit testing of the system. Paper 3, 4 and 7 have focused approach towards testing levels like Unit, integration, system and acceptance testing. Paper 7 and 8 discusses the importance of automation testing and compares various automation tools, whereas the book covers most of the concepts and along with it talks about the non-functional testing aspect as well which covers reliability, performance, stress testing techniques. Paper 1 has focus on Generic algorithm where test case creation, test data generation is automated.
Page left blank
Literature map
Concept matrix
Articles Black box : Boundary value, cause effect graphs White box ; loop , branch Grey box Testing levels unit , integration, system Testing goals functional Generic Algorithm Non functional Use case testing Automation testing State transition diagram Automation tools QTP, selenium, test complete
1 x x
2 x X x
3 x
4 x X x X
5 x X x
6 x x x x x x x x x x
7 x x x x X
8 x x
Discussion
Software testing is a vast field which is performed with the sole objective of finding defects in system under test. The question is to what all techniques can be used to test and the importance of these techniques is researched. Techniques like Black box, white box and grey box are prevalent to find the defects. Black box covers equivalence partioning, boundary value analysis, cause effect graphing to design better test cases, similarly white box testing technique focuses on loop testing, branch testing . Exploratory and use case techniques should be adopted and nonfunctional testing like reliability, security and performance still need deep focus. Genetic algorithm is being explored so that test cases and test data could be written automatically. Automatic testing with different tools is getting many takers in the industry. The different techniques are available to testers, developers to test the System and hence testers should focus on black box covering functional and nonfunctional whereas developers should focus on white box testing techniques.
Conclusion
Software testing is required to find defects in requirements, coding and design of the product under test. By software testing users tend to find how correct, complete, secure is there product and be assured about the quality of the product as compared to the specification. There are various techniques like Black box which covers equivalence partioning, boundary value analysis, cause effect graphing to design better test cases, similarly white box testing technique focuses on loop testing, branch testing . Exploratory and use case techniques should be adopted and nonfunctional testing like reliability, security and performance still need deep focus. Genetic algorithm should be explored so that test cases and test data could be written automatically. Automatic testing with different tools is getting many takers in the industry. There are different techniques available to testers, developers to test the System and hence testers should focus on black box covering functional and nonfunctional whereas developers should focus on white box testing techniques.
Limitations of this study that no experiments have been performed and most of the research is performed by reading articles and research papers.
References
Sharma, C., Sabharwal, S., & Sibal, R. (2014). A survey on software testing techniques using genetic algorithm. arXiv preprint arXiv:1411.1154.
Ghuman, S. S. (2014). Software Testing Techniques.
Meenakshi, D., Naik, J. S., & Reddy, M. R. (2014). Software Testing Techniques in Software Development Life Cycle. International journal of computer science and information Technologies (Published by IJCSIT), 5(3), 3729-3731.
Khan, M. E., & Khan, F. (2012). A Comparative Study of White Box, Black Box and Grey Box Testing Techniques. Editorial Preface, 3(6).
Chauhan, R. K., & Singh, I. (2014). Latest Research and Development on Software Testing Techniques and Tools.
Kaur, H., & Gupta, D. G. (2013). Comparative Study of Automated Testing Tools: Selenium, Quick Test Professional and Testcomplete. Harpreet kaur et al Int. Journal of Engineering Research and Applications ISSN, 2248-9622.