Test Planning Process (According to IEEE standard 829)
Jaspreet Kaur a, Gunjan Goswamib
a,bAssistant Professor, JSS Academy of Technical Education, Noida
After completing this chapter, students should be able to:
1. Illustrate the importance of software testing.
2. Identify the testing activities and deliverables of the testing process.
3. Demonstrate the test cases and test suits by using testing techniques.
4. Prepare test document according to IEEE standard 829.
Key terms: test plan, test approaches, entry & exit criteria, Estimation & Evaluation of project, licensed software, IEEE 829 Standard.
Test Plan preparation
Before staring the chapter we should be clear about the concepts of testing & planning of a software product. A brief introduction of terms related to software testing is given below.
1. Software Testing: It is a process of evaluating the software item to find out difference between given input and target output. Testing checks the quality of product. It should be started before the development of the product or in other ways it is verification & validation process of any software product. It measures the quality of product or service under test. The quality of the application can be constrained to common quality attributes like reliability, portability, maintainability, efficiency, usability ,reusability, and compatibility.
2. Verification: Verification is the process of evaluating the product that satisfies the conditions imposed at the start of the development phase. In other words, to make sure the product behaves the way we want it to verify and confirmation of product by evaluating the result as per requirements.
3. Validation: Validation is the process of evaluating the product that satisfies the specified requirements at the end of the development phase. In other words, validation is done to ensure that the product is developed as per customer requirements.
4. Process: Testing is a process rather than a single activity because this follows the whole software development life cycle.
5. Planning: Planning deals with what we want to do. As with planning we can control the various bugs and undesired activities that would be coming during testing. Project manager will plan the development of project based on requirement and features. The planning phase helps to find out the gap and inconsistencies. Once the planning is finished, then we are ready to move in to development phase.
Software Test Plan is a detailed document that outlines the test strategies, scope & objective of testing, functionality, test estimation & test deliverables. A test plan details about each step taken to achieve a certain result and states the objective of each action. The plan also shows the projected resources, risks, and personnel involved in the test. We should use a test plan if we are looking for elimination of bugs and errors in software before it is handed over to customers. We can say 'it is blue print to conduct different testing activities which is monitored & controlled by testing team & test manager. The test plan should include goal, scope & its schedule. To create a test plan following steps should be followed:
4.1.1 Scope of Test Plan:
Scope management defines the scope & objective of project. It includes:
1) Understanding what product constitutes at the time of release in market.
2) Breaking down all the important features of a product.
3) Prioritize the features for testing the product.
4) Deciding which features is to tested or which are not to be tested.
5) Gathering all the details to prepare estimation of recourses.
6) Calculate the man power to complete the project.
7) Check the time constraint to complete the project.
It is always better to list down feature & functions that we have planned to be tested like roles & responsibilities of every person involved in planning team, schedule, budget & timeline of project. Also, it is always good to start the project by knowing its requirements from end user & get a clear picture of the entire product. Some time it happen new features are to be added at planning phase. Knowing the features and understand them from usage perspective will enable the testing team to prioritize the features for testing. The prioritizing of the following features to be tested are:
1) Features that is new and critical to release
The new features result in new code & need attention to exposure of defects. Hence, it is required that these features are to be tested on priority basis in order to get prioritization right. Some customer, developer & testing team will identify & highlight new features.
2) Features whose failures can be catastrophic
Regardless a feature is important or not, any features which result in damage of whole the system must be avoided. (It may be due to unexpected hardware crash such as disk drive crash or due to software conflicts).
3) Features that are complex to test
Testing team helps to identify the features that are difficult to test. We can also prioritize this feature before actual development start & allocate the tools to check the consistency of these features.
4.1.2 Objective of the Test plan
1) Objective of the Test plan clearly defines the scope of test plan & what scenario should be tested & not to be tested.
2) It includes common testing modules like unit testing, integration testing, and acceptance testing (alpha & beta testing)
3)It should include the methods to eliminate /resolve the risk.
4) It should include the guidelines of the software project & deliverable for the client.
4.1.2 Features to be tested
Following features of software to be tested are:
1. List all the requirements (hardware or software) that are to be verified.
2. List out all the design specifications.
3. Mention the criteria that will be used to determine whether each test item of software has passed or failed testing.
4. List out testing tool required to perform testing.
5. Mention all test deliverables. (Test plan, test cases, test scripts, defects, test reports)
6. Provide cost & effort estimate for detail cost estimation of project.
7. List out responsibilities of all staff members.
8. Mention criteria to be used to suspend the testing activity.
9. Mention testing activities which must be redone when testing is resumed.
10. Mention the names and roles of all persons who must approve the plan
11. Mention the space for signatures and dates.
Features Not to Be Tested:
During the planning test manager will decide which features are not to be tested. This will help to balance the resources & requirements of a test plan so that Risk would not be faced during testing phases.
1) List the features of the software which will not be tested or not included in the version.
2) provide the reasons for not testing the mentioned features (like features of low risk).
Importance of Test plan
Test Plan helps the customer, developer, business manager to understand the need of test plan. It is like rule book which can be followed by everyone conveniently.
Test strategy is clearly defined so that it helps testing team & test manager. Test Plan helps to improve the quality testing product hence deliverable should be error free.
4.1.3 Setting up the Target Criteria
This part describes success criteria for evaluating the test results.
Success criteria (entry and exit criteria)
Entry and Exit criteria must be specified to start and end the testing project as it is important for the success of any project. If we are not clear with start and end criteria this means that our objective is not clear. By defining exit and entry criteria we define boundaries for development products. For instance, we can define entry criteria that the customer should provide the requirement document or acceptance plan. If these entry criteria are not met then project can not be started the project. On the other hand, while defining the exit criteria of the project one thing is to keep in mind that the customer has successfully executed or deployed he acceptance test plan. The entry & exit criteria must ensure the following things:
1) Technical requirements and statement is to be needed.
2) Test planning & feasibility should be cleared.
3) Required availability of software testing tool
4) Test case specification & test log must be cleared.
Plan should concise & specific & follow the IEEE guidelines.
Suspending Criteria describes the criteria that may result in suspending the testing activities and subsequently the requirements to resume the testing process. For example if 40% of test cases failed during testing, the the testing activities should be suspended & try to resolve the issues.
IEEE 829 Standard for Test Plan
IEEE refers to an international institution that define standards and guidelines which are globally recognized. IEEE has defined IEEE 829 standard for system & documentation. It specifies the format of a set of documents that are required in each stage of the software and system testing. IEEE provides eight stages in documentation process, producing a separate document for each stage.
4.2 Steps to write a test plan:
There are seven steps to be followed to write a test plan
1) Analyze the product & check its feasibility.
2) Develop test criteria
3) Resource planning.
4) Appropriate number of software license
5) Schedule & budget
6) Plan Test Environment
7) Test Deliverables
8) Process of generating Test Report
Fig 4.2 Steps to Write a Test Plan
NOTE: Test plan should always follow the IEEE standards so that all the features of test plan should be clearly defied before working on project
4.2.1 Analyze the product & check its feasibility:
The process of analysis start from software product requirement features. It is a process that describes features & functionality of a product .Analysis does a detailed study about the final product and its feasibility for development. The feasibility means the product should achieve its goals like minimum cost, completion of product with in time constraint, maximum quality, reliability, contribution of product to for environment sustainability, maintainability, usability, speed of recovery if software fails security etc.
If feasibility is done at developer end, then next step is requirement gathering. Analyst & tester talk to client or stakeholder about their ideas, on what platform they want to visualize the product or what kind of features they want to include in the software. To complete this features, SRS (software requirement specification) document is to be prepared by system analyst after the requirements are collected from various stakeholders like customer, end user & developer. The requirements coming to analyst are written in simple English language & then converted in to technical language.
SRS should consist of following steps:
An SRS should describe what the software will do & how it will do. It minimizes the time & effort required by developer to achieve the goal. A well defined SRS will interact with system hardware, other software & end user; parameters like response time, availability, portability, maintainability are all easily evaluated by SRS. SRS are described by the IEEE specification 830-1998. Following specifications should be followed to describe a SRS
a. Scope & objective of project: One of the most important items in the requirements specification is the precise scope & objective of definition of the project, its cost & schedule for completion of project, men power needed ,quality & what this project mean to society. So the impact must be checked.
b. Gather functional requirements from clients: Meet with the subject matter experts/clients to gather the requirements. Functional requirements specify the business requirements of the product in detail. Usually business requirements are described in terms of the actions that user performs on the software system. This is called the use case model. The requirements also described in terms of E-R Diagram and stored in the form of data dictionary.
c. External interface requirements: Software system interacts with number of application externally for input & output data requirements.
d. Non functional requirements: Non functional requirements describe how the software system should operate. It includes system performance, scalability of application, software security, maintainability, usability, availability, and logging and auditing of a system, data transfer requirements, multi lingual support etc. It is required to complete the SRS with all necessary peer review & discuss with managerial team.
To complete the analysis of a product, we should follow the following steps:
1. Identify the resource management
2. Identifying responsibilities, staffing & training requirements
3. Strategy for deciding test approach
4.Estimation of cost & schedule
5.Estimation of size & effort
6.Estimation of Risks & Issues
7.Administrating the Test Strategy
4.2.1 Identify the resource management:
While planning of a testing product, the project manager should provide the details estimate of various hardware & software resources. Some of points need to consider during resource planning:
1. Software configuration required like RAM, processor, hard disk, operating system etc.
2. Appropriate number of licensed software required to run the project.
3. Man power needed from start of project to end of project.
4. How much budget is required to complete the project?
5. Use a resource management tool or software for personal, team and business.
4.2.2 Identifying Responsibilities, staffing & training requirements
The project manager defines the role & responsibility of every personal involved in the project. Project manager will explain the specific role of a person based on their experience, skill, expertise or even based on certification. He summarizes the role & responsibility to conduct the project. The staffing management plan details the project human requirement & how these requirements are to be fulfilled.
Table 126.96.36.199: Identifying Responsibilities, staffing & training requirements
Role Team Responsibility Skills required Estimated start date Estimated duration Part time/full time Total time commitment
Role is defined by project manager only Identification of role of person in particular team In which team a employee lie like planning team ,development team, Testing team etc. Description of skill set required to complete the project Describing when resources are required on the project How long the resources are higher(calculate a rough estimate of employee time period on al project) Either person is working part time or full time on a project. What will to the total time employee spend on completion of project?
The staff training plan requirement start when the project manger finds the skill gap weather staff member requires any knowledge to complete the project or not. Providing training certification to staff members increases their skill set & help them to complete the project. The training requirements are specified in table below:
Table 188.8.131.52: Training Requirements
Role of person Type of training required Time span of training Cost estimation Resources or tool required for training
What type of role is providing to the employee? What kind of training is given to them? Is it product based or project based? What is the duration of training? Calculate the cost estimation for particular training How many resources are available with company or how much has to be hired.
4.2.3 Strategy for deciding test approach
It is a high level document which defines the basis of software testing approach. It is developed by test manager or business analyst to satisfy the business requirements. Test plan strategy should not be modified very frequently as it follows the approaches that are accepted in a proper planned way .Test strategy consist of the following parameters:
1) Scope & objective of test product: The objective of the business product and testing scope must be defined in test strategy.
2) Business issues : What is the budget of the project, how much time is needed for testing, how much test resources & tools are needed are the part of business issues which needs to be considered before the actual testing starts.
3) Testing approaches: What type of testing is needed to test a document like load testing, stress testing, smoke testing, system testing, functional testing etc. are all included in this section.
4) Test deliverables: It delivers the documents required from the testing team to developer, how they would keep the record of the testing tools & other resources and finally what the client obtained at end of day; all should be tracked in this section.
5) Defect tracking approaches: It deals with type of the testing tools that will be used for tracking the defects. To track the defects, testing team will communicate with development team and will decide how to track the defect, what steps would be needed to resolve the defect etc.
6) Training: If some new tools are introduced in the system, then provision of training is necessary given by the management team to testing team.
7) Automation: Automation means using an automatic testing to execute the test cases & test suite. An automation tool enters the test data, compare actual & expected result and generate a report.
8) Risks: What kind of risk are coming in the project & how these risks are resolved is also included in the document
Fig 184.108.40.206: Steps to develop Test Strategy
4.2.4 Estimation of cost & schedule:
Estimating cost or effort is one of the important tasks in test management. How long this testing will go on & how much it will cost. For small scale projects it is easy to answer these questions but for large project cost, effort & schedule estimation is tough.
Table 220.127.116.11: Example of Task, Members and Estimation effort
Task Members Estimation effort
Create the test specification Planning team 15 per-hour
Develop test design Designing team 12 per-hour
Perform coding & implementation Development team 10 per-hour
Check the product on different module testing Testing team 30 per-hour
Total effort estimation is summation of all effort that is 67 men-hours
Resources are the important factor that is required for completion of project resources it can be time, money, man power, technology & tools.
Cost: Cost means the project budget that is it shows the overall expenditure spend on a project. Cost can be calculated according to estimated size of the project. Suppose an average salary of a team member is 1000 rupees per hour & time required to complete the tasks are 200 hours. Then cost of project is 1000*200=200000 rupees.
If the project cost is accurate, then the project can be managed in a better way. Test estimation means breaking down the task in to small sub tasks, and then addition of the estimation of each task.
A schedule is considered to complete these tasks. The test manager needs several types of input before making the schedule.
1) Project estimation & project deadline for completion.
2) Numbers of employee on particular project.
3) How many resources are available with the team & how many are licensed & how many they have to hire.
Suppose the project deadline is one month then divide the whole tasks of the project in below tentative schedule. Sometime some tasks are complex so they take much time as compared to simpler tasks.
Create the test specification Start date 1/2/18 to 7/2/18
Develop test design From 8/2/18 to 15/2/18
Perform coding & implementation From 16/2/18 to 20/2/18
Check the product on different module testing From 21/2/18 to 28/2/18
4.2.5 Estimation of size & effort
To estimate size of software or lines of code required to develop software, generally we identify amount of new software's that must be developed. It also includes the functionality of pre existing component that are needed to be included in a new version. The size estimation is calculated from customer's requirement proposal or from SRS. All sizes are based on least, likely & most parameters. But the mostly used methods for size estimation are:
1) Expert judgment (Experts past experience).
2) Comparing old version with new version if exists.
3) Size of the database.
4) Checking the methodology by using pre existing algorithms or by automated tools to run the software.
5) Software cost Models (different model have different parameter for statistical analysis or rigorous calculation by mathematical notations).
Table 18.104.22.168: Estimation of size & effort
Total size estimates Least(LOC) likely(LOC) most(LOC)
Expert judgment 12000 15000 20000
Comparing old version with new version if exist 17000 18000 19000
Size of the database 9000 10000 12000
Checking the methodology 12000 13000 14000
Software cost Models 7000 8000 9000
Composite Least=7000 Likely=13000 Most=19000
It is required to track the accuracy of the estimates with actual as the project progresses & re- estimate the project size with actual data. To generate cost & effort, the whole process is breakdown in small sub process of software engineering activates.
1) Identify the sequence of activities of a project.
2) Divide the activities in to small modules or tasks that can easily measure.
3) Estimate the effort required to complete the task (number of person/days).
4) Combine all the efforts to calculate the project total effort.
5) Obtain cost or prize of each effort from cost database.
6) Computer the total cost & effort for each task.
7) Combine the entire task & calculate the cost & effort for entire project.
4.2.6 Estimation of Risks & Issues
Software development activity uses a variety of tools to code the project & clear knowledge of requirement & design analysis. But during development some uncertainty is created. This is known as software or project risk & the success of project depend upon how the risks are compressed or eliminated.
Risk is an unexpected loss that may or may not occur in future. Loss can be anything like development of poor quality product, not met with deadlines of project or sometime missing requirement from customer side. Software risks are of two types:
1) Internal risk: These risks are with in control of project manager, also known as controlled risk. All team members must be aware of risk. Such risks are explained during the planning of project
2) External risk: Risks that are not in knowledge of project manager & all team members are called External risk. These types of risks come from client side probably when client demands to work on new technology or tools to change GUI or database tec.
To estimate the risk following points should be clear:
1) Give a precise description of risk & its related issues.
2) What kind of loss that client or company bear when risk will occur?
3) Is there any method to suppress or remove the risk? If yes, then it should be clearly explained?
4) Explain how many chances (in percentage) of occurrence of risk.
Risk management team will estimate the risk by knowing the Risk identification, Risk analysis & Risk planning & monitoring.
4.2.7 Administrating the Test Strategy
Test strategy means what objective we want to achieve & what process & methods will be followed to achieve the goal. It is a quality assurance activity in software life cycle. The chances of any software activity missing is minimum, when we follow proper test strategy. The testing plan should be discussed with every team member before going to execute. According to IEEE Standard test strategy is a sub part of test plan. Hence all the details of these strategies must be defined in test plan. Test strategy defines guidelines how a test plan is to be followed in order to achieve objective & execution of test plan. Administrating a test strategy includes:
1. Scope & objective of testing software.
2. Reviewing about missing requirements.
3. Test methods to be used (unit testing, integration testing, system testing, acceptance testing etc.)
4 Test Environment set up (different for different team like user acceptance team, testing team etc.)
5. Testing tool (software tools for execution of project)
6. Bug removal & calculation of risk.
7. User acceptance testing (user approval on the project)
8. Review & summary of project before handed to customer or release in market.
NOTE: SRS is a document that clearly define software requirement, test strategy estimation of size, cost and schedule. Before building project SRS should be clearly defined.
4.3. Develop test criteria
To get outcome of test strategy, a team outlines entry & exit criteria of a software product.
4.3.1 Entry Criteria
As we have already discussed in brief about the Entry, exit & suspension criteria in section 4.1.3, here we are discussing more elaborated meaning of entry, exit and suspension criteria. Software testing life cycle needs to be clear about entry criteria test level (unit, integration, system & acceptance testing) distinctly, so that developer can be sure about user objective & requirement. If the user requirement is specified at entry level then chances of bugs would be less & project requirement would met soon. So, it is mandatory to mention all the features & constraints at entry level of test case. As entry criteria is finalized by testing team (controllers) & business team, it would be better to initiate the testing tasks. But if we ignore the entry criteria conditions, risk may be introduced. Before start the testing level the following features of entry criteria are must be taken care of:
22.214.171.124Test environment for Unit testing
1. The test environment has to be established like resources should be available, a tool has to be finalized, a team member task has to be given to them.
2. Planning, feasibility of product & requirement gathering should be completed.
3. Designing part like which design methodology is adopted (top down or bottom up) for running the project has to be completed.
4. System configuration (hardware & software configuration) should to be completed.
5. All coding singular modules must be available at the time of unit testing.
126.96.36.199Test environment for Integration testing
Test environment for integration testing includes:
1. Unit testing of all modules has to be completed before integration testing takes place.
2. Checking of risk or how the risk is to suppressed/resolved has to be done.
3. If any coding error found, that it has to be resolved before applying to integration test criteria.
4. Establishment of integration testing plan & test environment has to be completed before integrating the modules.
5. Modules dependency (coupling) & strength of between the modules (cohesion) must be verified before starting the integration testing.
188.8.131.52 Test environment for System testing
After the completion of integration testing entry & exit criteria, the next step is move further towards setting of environment. The same steps has to be followed in system testing like
1. Fixing or removal of bugs after integration testing
2. Checking of coupling & cohesion of modules.
3. Customer objective has to be met.
4. Test cases must be available for system testing.
4.3.2 Suspension/resumption Criteria
When condition of these criteria is met then testing team will suspend (suspension)/resumes (resumption) the process. The test process has to be resumed until the bug is not recovered. This criterion should be applicable when the following points are not in favor of software project:
1) Functionality of building project is not working according to requirements.
2) Hardware/software is not available at the time of project schedule.
3) The build up contains serious error or bugs while executing.
4) Test resource like tools are not available or older version is available.
5) Lack of training or skilled staff is not available.
6) Documentation is poor or manuals are not up to mark.
The testing is started only when the problems created during suspension criteria is resolved and typical defects are removed. Then it is called resumption of test product.
Exit criteria is analyzed & evaluated at the end of project life cycle, when we assure about project defects are removed from the software. It is the conditions & constraints that must be satisfied before closing the project. Testing should be stopped only when 100% customer requirement is satisfied & defect rate falls below the acceptable level. The exit criteria is done when the below major points have been covered.
1) All project modules are executed at least once.
2) All statement are covered & executed at least once.
3) All paths & code branches must be executed.
4) More focus given to higher priority modules (codes).
5) Regression test is done if needed at development time or at the end of project.
6). All major functional testing has done.
7). Time lines & budget is reached to limit.
8). All test items & deliverables are documented & reviewed.
Fig 184.108.40.206 Flowchart of exit criteria
220.127.116.11 Run Rate: Run rate calculates the number of test cases and test modules are running (executed) according to project requirement.
Percentage of test case executed= (number of test cases executed /total number of test cases)*100
18.104.22.168 Pass Rate: It is the percentage of test cases run without error from total number of test cases. The pass percentage much be 95% or above of a software project.
Percentage of pass rate= (number of test cases executed without error /total number of test cases)*100
22.214.171.124 Failure Rate: Failure rate calculates the number of test cases failed or having error according to test criteria guidelines.
Percentage of failure rate= (number of test cases executed with error /total number of test cases)*100
For better understanding, consider the following example:
Let us assume that there are 90 test cases for module A., B & C. Assume 30 test cases for A, 30 test cases for B & 30 test cases for C. In first release, out of 90 test cases 5 are failed and 85 are passed. Calculate the pass & fail percentage at first testing level of release. If out of 5, 2 bugs are fixed and are not fixed, in second testing level release, then again calculate the pass & fail percentage during system testing.
Solution: As out of 90 test cases 85 are passed at testing level 1,then pass percentage =85/90*100=94%
As 5 % test cases are failed at first level then fail percentage=5/90*100=5.55%
In second testing level those out of 5 bugs; 2 are fixed during system testing then pass percentage =87/90*100=96.66%
As 3% test cases are failed at first level then fail percentage=3/90*100=3.33%
In system testing cycle 1=run every module
In system testing cycle 2=re-run the failure module & run test cases which are modified during regression testing
In system testing cycle 3=re-run every module until 100 % run rate is achieved.
4.4 Resource Planning
Test plan ensures about resource requirement and equipment required which is necessary part of test planning process. Resource planning includes hardware, software, man power, test administrator & assurance of software quality, time & budget. It also includes the training & workshop needed for new developer & enhancement of quality needs of old staff, as it fills the gaps between available & expected skills.
4.4.1 Detailed Summary of Machine Configuration
Hardware equipment & software environment are required for testing the software. For example in the given below table hardware & software are listed out for running a project.
Test computer system Server required Software required Hardware configuration
Operating system: window XP/2000/linux etc. Oracle-SQL server, Java web server ,tomcat apache etc. Oracle, Java, PHP, Dream weaver etc. RAM size: like 8 GB of RAM (minimum), 32 GB for file storage, number of processor units (CPU'S) to execute the task.
4.4.2 Human Resource (estimation): Along with schedule, a good manpower is needed to complete the project. By, defining proper planning & tracking of project, project manager estimate the members of project team and define roles & responsibilities for every member. Calculation of average staff leaving the project & joining the project is also needed. More software project gone from the hands of company by lacking of man power. Estimation of effort is done by project manager with the help of designer, developer & tester. There are number of methods for effort estimation:
Fig.4.4.2 Resource estimation
1. Work break down structure (WBS) effort estimation : This method decomposes the team & accordingly works in to manageable chunks & assign task to very team. According to sub division of work team member are allotted for specific modules. This is the best way for effort calculation. Number of new members is allotted to the project as according to project requirement & break down structure. WBS defines the scope of project so that every team member can understand their tasks, review all the requirements from SRS & make sure they are added in WBS. WBS should satisfy the following criteria:
1) Every WBS has a clear outcome.
2) Every WBS has a well defied interface.
3) WBS has to follow divide & conquer rule.
4) Every WBS has to measure as a milestone for achieving success.
5) Every WBS has ability to measure skill set.
6) Linking of every WBS has to clear. It should maintain the property of low coupling and high cohesion.
Fig 126.96.36.199 Work Break Down Structure
2. Functional point estimation(size estimation): This is the most useful technique to calculate the size of the project. The primary goal of function point is to evaluate the system functionality from user point of view. It is a standard & fixed method to measure the functions of the software. To achieve this goal analysis is based on the ways user interacts with system. If the size of project is clear, we can easily calculate the man power & budget of the project. The major components of functional point estimation technique are:
1) Unadjusted data function point i) Internal logical files (ILF)
ii) External interfaces file (EIF)
ILF is a group pg logically related files or data that is maintained inside the user boundary.EIF is group of data or related files that are maintained for user reference and is a part of another application.
Fig 188.8.131.52 Relation between ILF & EIF
User input is the data that comes in user boundary for evaluating the application output. User output is data that is provided by the user to outside world. User inquires database is direct information system about stored data that is provided to users when they request for specific files & then data is provided to user with no manipulation.
The five function points are ranked according to their complexity: low, average or high. After defining all the functional points; UFP (unadjusted function point) is calculated under predefined weight of each functional point.
UFP CALCULATION TABLE
The last step involves the processing complexity of the project or application. In this part impact of 14 general characteristics are rated on a scale of 0 to 5 in term of feedback of application or project.
Table 184.108.40.206 UFP CALCULATION TABLE
Total Unadjusted functional point Z1+Z2+Z3+Z4+Z5=Z6
VALUE ADJUSTMENT FACTOR CALCULATION TABLE:
GENERAL SYSTEM CHARACTERSTIC DEGREE OF INFLUENCE(DI)(0-5)
1.Data communication Data communication done in project is sent or received by using communication facilities.
2. Distributed data processing How data is processed& distributes over the network
3. Performance Throughput or response time of application in term of design & development of project.
4. User configuration Configuration used for running the application
5 Transaction rate What are transfer rates of an application? (calculate on weekly or monthly basis)
6 Online data entry How the information is entered by online user & how it calculated on application part?
7 End user efficiency Is this application is beneficial for customer?
8 Online update How frequently application updates are donr by online user.
9 Complexity processing Measurement of complexities of application by checking logical & discrete components.
10 Reusability Application design should be reusable to other application
11 Installation How installation & deployments is done after completing the application.
12 Operational ease How easy it is to user the application.
13 Multiple sites Is application is customer based or multiple organization based.
14 Any other change need to facilitate Is editing option is included to application for end user purpose.
VAF (TOTAL DI*0.01)+0.65
Function type ESTIMATED COUNT WEIGHT FP COUNT
UI 24 4 96
UO 16 5 80
UE 22 4 88
ILE 4 10 40
EIF 2 7 14
TOTAL UFP COUNT 318
GENERAL SYSTEM CHARACTERSTIC DEGREE OF INFLUENCE(DI)(0-5)
1.Data communication 2
2 Distributed data processing 0
3 Performance 5
4 User configuration 5
5 Transaction rate 2
6 Online data entry 4
7 End user efficiency 3
8 Online update 5
9 Complexity processing 4
10 Reusability 5
11 Installation 4
12 Operational ease 3
13 Multiple sites 4
14 any other change need to facilitate 5
VAF (TOTAL DI*0.01)+0.65=1.17
Disadvantages of function point:
1) Identify function points and assigning weights to them requires more training as it is more than counting lines of code.
2) It cannot be applied to all type of projects as less research data is available for function point rather than LOC
3) FP's are not applicable to system programming area where data structure manipulation id is required.
4) it is a time consuming method.
5) it has low accuracy rate as more calculation is required to judge the 14 parameters.
3. Three point estimation(schedule estimation): This is the best technique for developing estimates of project with team members. It is useful tool to estimate the task for project manager & developer. Three values are produced for task that is allotted to team members:
1. Best estimate (if the task takes minimum time)
2. Average estimate (if the task takes moderate time)
3. Worst case estimate (if the task is performed more than 100 times )
For example we think that best estimate is when the project is complete with in one month or the time prescribed by the customer by listing all the requirements with positive risks and average estimate when the task take 2-3 months & more time spending on calculating risk & understanding the requirement. The worst scenario is when the project takes 6 months by spending more time on resolving bugs & suppressing negative risks as according to project requirements. Two popular formulas for estimate calculation are:
//Plz write about Triangular distribution & Beta distribution//
1)Triangular distribution E=(best estimate +avg. estimate +worst estimate)/3
If Best estimate=4 months, average estimate=7 months, worst case estimate=15 months
E= (4+7+15)/3=8.66 months
2)Beta distribution or PERT Distribution= E=(best estimate +4*avg. estimate +worst estimate)/6
E= (4+4*7+15)/6=7.83 month
Metrics for estimation: For any type of estimation, metric is required to calculate the effectiveness of the project. At the end of project actual value of estimation is compared with target value to find out the difference in exact estimation.
To calculate effort estimates = (planned effort- actual effort)/planned effort*100
Size estimate= (planned size- actual size)/planned size*100
Schedule estimate= (planned schedule- actual schedule)/planned schedule*100
Cost estimate =number of person-months* Rs (value per person month)
NOTE: Two Estimation are very important for calculating size of product are LOC/KLOC and function point Estimation.
Test administrator is the person who coordinates day to day activity of development, supervision of testing staff & activities, follow the procedure & guidelines of project, establish contact with other software companies in order to maintain & supply testing materials. Major roles & responsibility of test administrator are:
(1)Coordinate with the members of academics; provide certification for courses & training.
(2)Administrate the centralized team that deal with development, installation, hardware, software & peripherals.
(3)Provide routine directions and information to training students regarding polices & procedures of testing project.
(4)To maintain the corporate work environment.
(5)Organize & coordinate training & skilled programmer for employers.
(6) Checking employer performance & development and accordingly plan the system, processes & facilities for them.
(7) Ability to have data management skills
(8)Have ability for work allocation, hiring & firing the employee.
(9) Keeps updated information regarding new technology & tools & update the staff & company accordingly.
(10)Maintain confidentiality of test result & recommendation & security of testing materials.
4.4.3 SQA Members
The role of SQA team is to ensure that product does not deviate from original requirements & design specifications. If this is discovered & notified, and then SQA team will alert the development team not to further divert from requirements & action plan. Even SQA team will walkthrough & review every phase at a particular stage to observe & correct the modules. SQA team leader provide the flow of information from SQA team to software engineer. SQA leader ensure that software engineer confirm to standard of SRS. SQA leader prioritize the defects provided by software engineer, he will review & provide the solutions to solve the defects. SQA Activity include SQA meeting & review report.
The following activities done by SQA members are:
1. Process improvement planning
2. Conduct of audits of projects ,services & modules
3. Check or review on development, design & modification modules.
4. Provide training to different team on different tools as they required.
5. Metric analysis is done by SQA leader.
6. More focus on software process rather than product.
7. Follow the SQA standards that are pre-written.
8. Preventing the quality problems from occurring in software product.
Quality assurance is done by following steps **
(1)Develop Quality plan: People in SQA team play an important role. Thus, it is required to identify the roles and responsibility of every SQA team members. Every team member has to ensure that product met the entire software requirement as according to customer needs. Quality team coordinates with team members & higher authority team members.
(2)Focus on developing constraints: Time, money and effort are equally important constraints for quality assurance. By analyzing the cost of product, the revenue & return given by the project can be calculated. Cost includes training the team, review of code by higher authority & tools higher temporarily for testing purpose & if software or module fails then the cost include for recovery & back up also included. To achieve this more efforts are required with in time constraint.
(3)Defect management approach: This approach deals with finding out defects of projects & try to resolve it completely. Sometime it is not possible to remove all the defects, at this level try to minimize the defects. For this, try to alter the process steps, do changes in modules, try to maintain minimum coupling between dependent modules.
(4)Major software quality attributes: Major software quality attributes are functionality, reliability, maintainability, efficiency, portability, usability.
(a)Functionality: It includes sustainability, accuracy, interoperability & security.
(b)Reliability: It includes maturity & recoverability of software.
(c)Maintainability: It includes analyzability, changeability, stability & testability of software.
(d)Efficiency: It depends up on the system architecture & coding platform used.
(e)Portability: Adaptability or coupling the system so that the code run on one platform can easily be execute on another platform without the need of compiler.
(f) Usability: Usability includes the market value of the product after developing it and it also specifies how much profit we will get in return.
(5)Software quality assurance audit & review: Reviews are done to assess the quality & design of the software product. It is important to find and analyze defects in early stage of software development. The review & audits are done by technical team so that on every phase risks & bugs can easily monitor & resolved. The quality insurance auditor checks the results of reviews and resolves the quality related issues and finally generate a report on product quality.
Fig 4.4.3 Quality Insurance steps
4.5 Appropriate number of software licenses
Selecting an appropriate license scheme is crucial for the success of a project. As, it reduces the testing efforts & help to capture the market value. In market many open source software tools are available with code in which editing can be done over original design; but open source tools are not used for commercial purpose. So companies always prefer licensed software tools & schemes. Depending on the license scheme software, product that is going to sale in market sometimes may be completely attractive or sometimes it's useless. It is required to know completely which license scheme is beneficial for software while making a test plan. Some license scheme are organizational based, some are particularly for single user, some are free licensed with reduced features etc. but we have to find the best licensed software as per user requirements.
Note: Quality assurance is the process that assures the delivery of desired quality to the customer. So it always follows the quality standards.
Selection of licensed software
Success of business depends up on how licensed software's are being chosen.
Marketing tasks depend up on the licensed software as most of the good application becomes only successful if they are secure & verified by licensed software.
There are two license terms available in market
i) Perpetual license: Perpetual license is given to the software for its indefinite purpose or lifetime.
ii) Time'based: Time-base licensed software means that software will function for a short period of time. When the time expires the software license needs to be renewed.
Types of software license
1. Transportable: When we want to move the license between computers, but license work at one computer at a time. License is locked on a particular computer when it is installed by using keysight software manager online tool, we can transfer the license from one to another computer in small time span of 60 to 90 seconds.
2. Node-locked (fixed): This type of license is fixed and locked for a single computer as per the user needs.
3. Floating/network: This license is server based license; once installed on server it can be connected to multiple computers that are linked to particular server.
There are more than 100 licensed of free ware tools are available in marker.
Some of licensed tools are: LoadUi, Loadster, loadImpact, Wapt, Telerik Test Studio, Tsung, Aglieloadsandstorm, Load2test, Blazemeter, Apploader, Vperforer etc.
4.6 Plan Test Environment
The test environment consist of hardware, software, network configuration testing tools, testing team and testing location to execute the test cases & test suites. The environment is configured as per the testing requirements.
4.6.1 Set up the test environment
Sometimes the testers face many challenges while doing the testing because the environment set up is not proper as per the software requirements. To remove the challenges faced by testing team set up environment must consists of:
i) Understand the test requirement & done set accordingly
ii) Which browser version is needed to the software?
iii) Which operating system is compatible as according to software product execution?
iv) Network (LAN, WAN, Connecters, bridges, routers, gateways, internet setup etc.)
v) Configuration of server for database connectivity & to support application.
vi) Platform needed to make front End (ASP, JAVA, VB.NET etc.)
vii) Data sets needed to test the data by using test cases.
viii) Documentation required like installation guide, user manual etc.
ix) Environment must be set up under the invigilance of tester, administrator and developer.
x) Licensed tools should be checked before connectivity of environment is to be done
4.6.2 Create test cases & test suits
Test case is an important part of software testing without which it is not easy to track the reasons of bugs and failure. It is a step by step procedure of the actions that we are going to execute the test data. A test case consists of set of input values, precondition, expected results or output values and post condition that check the requirements of an application meet or not. As an input it also checks the working conditions of a module. The test case template is shown in Table 220.127.116.11
Table 18.104.22.168: The Test Case Template
1) Test case id Id of test case.
2) Test case objective What the test case will do?
3) Requirements of test case Describes all the essentials to run the test case.
4) Precondition Constraints that is applied to execute test case.
5) Test procedure A step by step procedure or algorithm to implement test case.
6) Test data Data set that is to be used to test the test case.
7) Actual result The result that is to be obtained after testing the data.
8) Expected result Actual output should match with target results.
9) Status Pass or fail of test case.
10) Remarks Comments on test case if needed.
11) Created by. Who created the test case? Mention the name of author
12) Date of creation On which date test case is created.
13) Date of execution. When the test case is executed. Mention the date of execution
14) Environment set up The hardware/software tools are needed to execute the test case.
15) Risk analysis This has to be identified during testing the test data.
Test suite: It is a container for test cases, where series of test cases combined to produce a test suite.
Example For making the online transaction of money through debit/credit, user has to enter right card number, cvv number, date of expiry of card and amount that has to pay by the user. These all steps are the combinations of test cases that user has to successful run and combination of all test cases become a test suite. A test suite can contain 100 or even 1000 test cases for execution.
4.6.3 Testing & Deployment of test environment: After testing all the modules of software product by using test cases and test suites, the last step is the deployment of test product in standardized test environment As the internet is very fast today, so as soon as the project is completed and verified by company and customers the project is deployed on customer site and company members start using it. But still sometime there is delay between the actual completion of the project and software reaching the customer.
The steps involved until deployment are:
1) The software product organization make the product available on master server so that in any case copy of the product can be made.
2) The customer would be informed about the product availability & product completion.
3) Then customer would place an order for the software product to software organization.
4) If all the pre requisite conditions are met from customer side like licensing, payment, security etc. then product can handed over to customer.
5) Lastly a customer receives and installs the software product.
Currently three models are followed for deployment purpose:
Note: Tester generally makes test cases based on the requirement of customer. Problem is that requirement usually defined at initial of project & sometimes test cases are being made at later stages like designing, testing, verification etc.
1) Traditional media install deployment model: This model is expensive and time consuming. The product had to installed at customer site & customer required to pay for hardware and installation cost.
2) Web based deployment model: This is a self service model. The software is placed on web site and any interested user can buy it (purchase license) by doing payment. Customer log on the published web site, enter the details about company, purchase it and after that easily download and install the product. In this manner shipping cost and cost of installation at customer site is reduced and software is being available at customer site.
3) Hosted model: In this model, the ASP (application service provider) is used that obsolete the need for downloading the software. When updating is needed, it is directly updated on ASP site & user directly got the version on websites. This reduces the hardware & manpower cost. Even it reduces the product delivery cost and deployment time because product easily available to customer. In this way quality of product is also increased significantly as the customer is providing feedback on every product and its versions.
4.7 Test Deliverables: Test deliverable is a product, data or test script that is given to the customer at the end of project or at middle level after completing the different modules like SRS (software requirement specification), SDD(software design document) etc. so that customer can give feedback on each deliverable and accordingly changes can be reflected in the software product. Some test deliverable are provided before the testing phase , some are provided during testing and at last final product is handed over to customer when all test cases complete and product must be error free.
4.7.1 Test Deliverables before Testing
The components that are included before to start testing are:
1. Test plan
2. Necessity of test plan
3. Objective & scope of test plan
4. Entry & exit criteria of testing
5. Testing strategy
6. Testing Environment
7. Hardware & software requirement of testing
8. Installation guide /Environment
9. Test standards & guidelines.
10. Risk analysis
4.7.2 Test Deliverables during Testing:
The components that are included during the testing are:
1. Test specification document generated & reviewed from customer
2. Test case generation
3. Effort estimation (size, schedule & budget)
4. Man power required for testing
5. Dead line should be cleared accordingly scheduling plan has to be divided
6. Module clarity & platform required for development
7. Availability of Hardware & software tools
8. Data set required for testing.
9. Test metrics (like product & process metrics should be cleared)
10. Traceability of developing product
11. Test scripts (Manual or automated)
12. Calculation of risk & action taken to suppress it.
13. Product quality assurance
14. Security features
15. Authorization & authentication
4.7.3 Test Deliverables after Testing:
The components that are included after the testing are:
1. Find defects in software products
2. Verify software product so that it should fulfill user requirements
3. Check & improve software quality
4. Fixing & minimization of risk
5. Calculate product development cost
6. Check security features
7. Generate test reports
8. Take decision on software delivery
Note: During project test deliverable is given to stakeholder after every phase of SDLC life cycle.
4.8 Process of generating Test Report: After completion of testing, the last step is to generate test report. A Document has to be prepared & planned when testing is completed which comprises of testing activities, testing assessment and testing results is called test report. A test report should be clear, concise, short and specific and follow the report standards & guideline. The test report contains the following contents:
1) Purpose of the test report
2) Introduction of software project or product
3) Scope of test plan & testing (Item that is to tested or not to be tested should be included in scope)
4) Details of testing data used for verification of software product.
5) Metrics has to be prepared for test plans in graphical form like bar chart or pie chart, graphs etc. (How many test cases were planned, how many were executed, how many were pass and fail)
6) Number of defects observed, status of the defects & what action to be taken on resolving the defects. The project team will send the defects information to project manager like fixing of defects and defect density (number of defects /LOC)
7) Testing tools in involved & their specification
8) What kind of testing is performed like unit testing, integration testing, system testing, acceptance testing etc)
9) Software and hardware configuration must be included
10) Exit criteria are verified like all defects are resolved, verified and finally deleted.
11) Results of testing performed and conclusion has to be cleared.
12) Launching of the software product in market (clear market strategy)
13) Recommendations if required
14) Acronym & abbreviations
//** Please include the conclusion/summary of 4-6 lines from any book**//
Multiple choice questions:
Q: 1 what is the final outcome of the requirement analysis and specification phase?
1) Coding 2) software design document 3) SRS 4) User Manual
Q: 2 who write the SRS?
1) System Developer 2) system tester 3) system analyst 4) none of these.
Q: 3 testing can be applied to '..
1) Requirements 2) design 3) testing 4) coding
Q: 4 which of the following is performed by user?
1) Acceptance testing 2) unit testing 3) integration testing 4) system testing
Q: 4 Risk analysis of a project is done in:
1) System analysis phase 2) feasibility study 3) implementation phase 4) maintenance phase
Q: 5 Test Deliverables after Testing is composed of '.
1) Test plan 2) test design 3) verification of software product at final stage 4) fixing of error
Q: 6 which deployment model is used currently for Testing & Deployment of test environment?
1) Traditional media install deployment model 2) Web based deployment model
3) 3. Hosted model 4) none of these
Q: 7 Total quality Management is focus on''
1) Customer 2) Employee 3) supplier 4) all of these
Q: 8 service dimensions that would be imported are'..
1) Deliverables 2) inputs 3) tangibles 4) durability
Q: 9 Effort estimation includes''..
1) Size 2) cost 3) effort 4) None
Q: 10 A set of inputs, execution pre -conditions and expected outcome is known as
Test plan 2) test case 3) test document 4) test suite
Fill in the blanks.
1) The ASP (application service provider) is used to obsolete the need for '''.the software.
2) It is a container for test cases, where series of test cases combined to produce a''''. .
3) A Document has to be prepared & planned when testing is completed is called''''..
4) Time-base licensed software means that software will function for a'''''. When the time expires the software license needs to be renewed.
5)''''. are done to assess the quality & design of the software product.
6) The role of SQA team is to ensure that product does not deviate from '''''''
7) If the size of project is clear, we can easily calculate '''..of the project.
8) Verification is the process of evaluating the product that satisfies the conditions imposed at the ''''''.
9) Exit criteria is analyzed & evaluated at the end of project life cycle.
10) Test Plan helps the ''''''to understand the need of test plan
Very short answer question:
Q:1Define software testing.
Q: 2 Illustrate the term Entry criteria.
Q: 3 what is outcome of test deliverable?
Q: 4 How test environment is used for system testing.
Q: 5 Define scope & objective of test product.
Q: 6 How a failure rate is calculated in testing of product.
Q: 7 Differentiate between verification & validation.
Q: 8 why we use test report for a software product.
Q: 9 Name the factors of software estimation.
Q: 10 why licensed software is always preferable?
Short answer question:
Q:1 What are the factors to create a test suite?
Q: 2 Differentiate between entry & exit criteria of a test plan.
Q: 3 why do we use work breakdown structure in software building?
Q: 4 list the components of test deliverables.
Q: 5 Define the steps of software quality assurance with neat diagram.
Q: 6 Explain the different kinds of Risk.
Q: 7 why SRS document is important for collecting software requirements?
Q:8 Explain the Strategy for deciding test approach.
Q:9 why should we use test environment for testing a product.
Q:10 Explain the role of project Manager while working in a project.
Long question answer:
Q1:How a functional point estimation is done? Explain the factors of calculating functional point estimation?
Q: 2 Give a brief on major software quality attributes.
Q: 3 How the analyzes of a product is to be done. Give complete description of every step.
Q: 4 Brief with diagram the exit criteria of software testing.
Q: 5 Explain test deliverable before testing, during testing & after testing.
Q: 6 Draw & define the test case template used during software testing.
1.'Srinivasan Desikan,Gopalaewamy Ramesh',software testing & principle.
2. 'Gopalaewamy Ramesh',Managing Global software projects.
...(download the rest of the essay above)