The Goals of Bug Writing
As with all other forms of writing, it‘s important to remember who your audience is. Usually, the bug will be written for developers and for other QA engineers. Know also that other departments may need to understand the bug: marketing, tech support, etc. You can assume that everyone who may read the bug will have some understanding about the product which the bug is logged against and the technology used. They may not, however, know much about your specific test area.
1. Eliminate basic questions that a Development Engineer might have by including essential information.
2. Understand your audience: make steps understandable for other departments (tech support, marketing) or other testers not in your area.
3. Make searching easier for yourself and others.
4. Write in a way that demonstrates you‘ve done necessary isolation.
Title
Keep the title short and sweet. It should be as explicit as possible in as few words as possible. Think again about what the title will be used for and who might use it. A QA engineer might be looking to match their bug against what‘s already logged and needs a quick way to scan through the titles. Distill each bug to its crucial elements and put that in the title.
1. Try to write in a cause-and-effect manner (―When A is done, B happens,‖ or ―B happens when A is done.‖)
2. Avoid ambiguous wording, such as ―feature is broken/incorrect behavior/does not work,‖ etc. Instead, say precisely how it‘s broken, incorrect, or not working.
3. Use keywords that will make searching for the bug easier for yourself or someone else looking for duplicate bugs. Avoid using jargon, slang, or vocabulary that is too specific to your area.
4. When an assert appears in your bug, include the assert or a portion of the assert in the title(e.g., ―Assert, ‗index < fLength‘ when pasting text into very small text frame‖).
Description
This is where all of the information, the body of the bug, resides: Steps to Reproduce, Actual Results, Expected Results, and any other helpful or vital information regarding the bug.
Steps to Reproduce Bug reports usually suffer from two deficiencies:
1. Too many steps, often poorly organized. Having too many steps in a bug report makes it difficult to read and understand, especially given the confined viewing area in Vantive.
2. Too little information in a bug often leads to unnecessary extra effort and time. Often a bug will be sent back by the engineer as Cannot Reproduce as a result of the inability to follow poorly constructed steps in a valid bug. This can also indicate that you haven‘t completed enough isolation steps.
Essential information to include in Steps to Reproduce:
1. Setup variables: Indicate which printers, fonts, or drivers are necessary to reproduce this bug. Indicate the working OS, if it‘s essential information for reproducing this bug.
2. Environmental variables. For example, indicate if you are working in application or document mode.
Software Testing Training
Our Software Testing Partner
Software testing institute
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Thursday, December 18, 2008
Test Management System - test design and procedure development
Test Management System
Test design and procedure development
At the test design stage, testers will create a description of each test, document its scope and objective, and include any information that helps illustrate the purpose of a specific test such as requirements documents, functional specifications, etc. During the test development phase, testers will document detailed test execution steps and define the expected results for each step. The test management system will help in defining and documenting test cases by providing standard Web-based, pre-formatted template forms with fields based on the product and component information for editing and creating test cases. These get posted to the centralized database. This enables standardization and consistency across the testing team. It will also help in linking to the requirements specification to ensure traceability and test coverage. The test case may have been created due to a known defect and gets an association created with that defect. We can define the sequence in which test cases should be executed. This may be based on functional dependencies or some other factors like risk and other priority.
Organization
To verify application functionality and usability, tests have to realistically emulate end-user behavior. To achieve this, test execution should follow predefined logic, such as running certain tests after other tests have passed, failed, or been completed. For example, a user logs into the system, enters a new order and then exits the system. To emulate this simple business process, it makes sense to run the tests following the exact same sequence: log in, insert order, log out. The execution logic rules should be set prior to executing the actual tests.
Review
Once the test cases have been created, we can get them reviewed by required team members and customers. It is easy to communicate the test cases to the team because of the Web interface. This will verify the test cases developed by the test team and improve them further if required before actual testing begins.
Execution
The test cases can be accessed from any computer over the intranet/Internet, depending on how the test management tool is deployed. The test management system will help in locating a test case and provide a Web interface to process the test case. As the test is processed, the tester can immediately log the actual results along with pass/fail results and additional comments. The Web-based process supports parallel execution of test cases by many team members, which is not possible with a single flat file that gets "routed" around. If the test ever fails, it has an associated defect number which the tester can look at to see if a previous defect report should be opened, or a new one created. This helps to ensure that nothing falls through the cracks.
Maintenance
The test management system will maintain an accurate history of each run, including execution configuration, date and time of run, who ran the test, and any defects that were uncovered during the run.
Defect management
When a test case fails, the tester can enter the ID of the defect that caused the case to fail. The defect is, of course, inserted into the defect tracking system. The defect can be linked with the test cases. This will provide information to reproduce and analyze the defects.
Software Testing Training
Our Software Testing Partner
Software testing institute
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Test design and procedure development
At the test design stage, testers will create a description of each test, document its scope and objective, and include any information that helps illustrate the purpose of a specific test such as requirements documents, functional specifications, etc. During the test development phase, testers will document detailed test execution steps and define the expected results for each step. The test management system will help in defining and documenting test cases by providing standard Web-based, pre-formatted template forms with fields based on the product and component information for editing and creating test cases. These get posted to the centralized database. This enables standardization and consistency across the testing team. It will also help in linking to the requirements specification to ensure traceability and test coverage. The test case may have been created due to a known defect and gets an association created with that defect. We can define the sequence in which test cases should be executed. This may be based on functional dependencies or some other factors like risk and other priority.
Organization
To verify application functionality and usability, tests have to realistically emulate end-user behavior. To achieve this, test execution should follow predefined logic, such as running certain tests after other tests have passed, failed, or been completed. For example, a user logs into the system, enters a new order and then exits the system. To emulate this simple business process, it makes sense to run the tests following the exact same sequence: log in, insert order, log out. The execution logic rules should be set prior to executing the actual tests.
Review
Once the test cases have been created, we can get them reviewed by required team members and customers. It is easy to communicate the test cases to the team because of the Web interface. This will verify the test cases developed by the test team and improve them further if required before actual testing begins.
Execution
The test cases can be accessed from any computer over the intranet/Internet, depending on how the test management tool is deployed. The test management system will help in locating a test case and provide a Web interface to process the test case. As the test is processed, the tester can immediately log the actual results along with pass/fail results and additional comments. The Web-based process supports parallel execution of test cases by many team members, which is not possible with a single flat file that gets "routed" around. If the test ever fails, it has an associated defect number which the tester can look at to see if a previous defect report should be opened, or a new one created. This helps to ensure that nothing falls through the cracks.
Maintenance
The test management system will maintain an accurate history of each run, including execution configuration, date and time of run, who ran the test, and any defects that were uncovered during the run.
Defect management
When a test case fails, the tester can enter the ID of the defect that caused the case to fail. The defect is, of course, inserted into the defect tracking system. The defect can be linked with the test cases. This will provide information to reproduce and analyze the defects.
Software Testing Training
Our Software Testing Partner
Software testing institute
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Bug Logging , Writing , Reporting
Bug Logging / Writing / Reporting
The first aim of a bug report is to let the programmer see the failure with their own eyes. If you can't be with them to make it fail in front of them, give them detailed instructions so that they can make it fail for themselves.
In case the first aim doesn't succeed, and the programmer can't see it failing themselves, the second aim of a bug report is to describe what went wrong. Describe everything in detail.State what you saw, and also state what you expected to see. Write down the error messages, especially if they have numbers in.
When your computer does something unexpected, freeze. Do nothing until you're calm, and don't do anything that you think might be dangerous. By all means try to diagnose the fault yourself if you think you can, but if you do, you should still report the symptoms as well.
Be ready to provide extra information if the programmer needs it. If they didn't need it, they wouldn't be asking for it. They aren't being deliberately awkward. Have version numbers at your fingertips, because they will probably be needed. Write clearly. Say what you mean, and make sure it can't be misinterpreted. Above all, be precise. Programmers like precision. Useful bug reports are ones that get bugs fixed. A useful bug report normally has two qualities:
1. Reproducible. If an engineer can't see it or conclusively prove that it exists, the engineer will probably stamp it WORKSFORME or INVALID, and move on to the next bug. Every relevant detail you can provide helps.
2. Specific. The quicker the engineer can isolate the issue to a specific problem, the more likely it'll be expediently fixed. If you're crashing on a site, please take the time to isolate what on the page is triggering the crash, and include it as an HTML snippet in the bug report if possible. (Specific bugs have the added bonus of remaining relevant when an engineer actually gets to them; in a rapidly changing web, a bug report of "foo.com crashes my browser" becomes meaningless after the site experiences a half-dozen redesigns and hundreds of content changes.)
Software Testing Training
Our Software Testing Partner
Software testing institute
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
The first aim of a bug report is to let the programmer see the failure with their own eyes. If you can't be with them to make it fail in front of them, give them detailed instructions so that they can make it fail for themselves.
In case the first aim doesn't succeed, and the programmer can't see it failing themselves, the second aim of a bug report is to describe what went wrong. Describe everything in detail.State what you saw, and also state what you expected to see. Write down the error messages, especially if they have numbers in.
When your computer does something unexpected, freeze. Do nothing until you're calm, and don't do anything that you think might be dangerous. By all means try to diagnose the fault yourself if you think you can, but if you do, you should still report the symptoms as well.
Be ready to provide extra information if the programmer needs it. If they didn't need it, they wouldn't be asking for it. They aren't being deliberately awkward. Have version numbers at your fingertips, because they will probably be needed. Write clearly. Say what you mean, and make sure it can't be misinterpreted. Above all, be precise. Programmers like precision. Useful bug reports are ones that get bugs fixed. A useful bug report normally has two qualities:
1. Reproducible. If an engineer can't see it or conclusively prove that it exists, the engineer will probably stamp it WORKSFORME or INVALID, and move on to the next bug. Every relevant detail you can provide helps.
2. Specific. The quicker the engineer can isolate the issue to a specific problem, the more likely it'll be expediently fixed. If you're crashing on a site, please take the time to isolate what on the page is triggering the crash, and include it as an HTML snippet in the bug report if possible. (Specific bugs have the added bonus of remaining relevant when an engineer actually gets to them; in a rapidly changing web, a bug report of "foo.com crashes my browser" becomes meaningless after the site experiences a half-dozen redesigns and hundreds of content changes.)
Software Testing Training
Our Software Testing Partner
Software testing institute
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Automated Testing - Manual Vs Automation
Automated Testing - Manual Vs Automation
If you‘re only going to run the test one or two times or the test is really expensive to automation, it is most likely a manual test. But then again, what good is saying ―use common sense‖ when you need to come up with deterministic set of guidelines on how and when to automate?
Pros of Automation
If you have to run a set of tests repeatedly, automation is a huge win for you
It gives you the ability to run automation against code that frequently changes to catch regressions in a timely manner
It gives you the ability to run automation in mainstream scenarios to catch regressions in a timely manner
Aids in testing a large test matrix (different languages on different OS platforms).
Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.
Cons of Automation
It costs more to automate.
Writing the test cases and writing or configuring the automate framework you‘re using costs more initially than running the test manually.
Can‘t automate visual references, for example, if you can‘t tell the font color via code or the automation tool, it is a manual test.
Pros of Manual
If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
It allows the tester to perform more ad-hoc (random testing). More bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.
Cons of Manual
1. Running tests manually can be very time consuming
2. Each time there is a new build, the tester must rerun all required tests - which after a while would become very mundane and tiresome.
Other deciding factors
1. What you automate depends on the tools you use. If the tools have any limitations, those tests are manual.
2. Is the return on investment worth automating?
3. Is what you get out of automation worth the cost of setting up and supporting the test cases, the automation framework, and the system that runs the test cases?
Criteria for automating
There are two sets of questions to determine whether automation is right for your test case: Is this test scenario automatable?
1. Yes, and it will cost a little
2. Yes, but it will cost a lot
3. No, it is no possible to automate
How important is this test scenario?
1. I must absolutely test this scenario whenever possible
2. I need to test this scenario regularly
3. I only need to test this scenario once in a while
If you answered #1 to both questions – definitely automate that test
If you answered #1 or #2 to both questions – you should automate that test
If you answered #2 to both questions – you need to consider if it is really worth the investment to automate
What happens if you can‘t automate?
Let‘s say that you have a test that you absolutely need to run whenever possible, but it isn‘t possible to automate. Your options are
Reevaluate – do I really need to run this test this often?
What‘s the cost of doing this test manually?
Look for new testing tools
Consider test hooks
Software Testing Training
Our Software Testing Partner
Software testing institute
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
If you‘re only going to run the test one or two times or the test is really expensive to automation, it is most likely a manual test. But then again, what good is saying ―use common sense‖ when you need to come up with deterministic set of guidelines on how and when to automate?
Pros of Automation
If you have to run a set of tests repeatedly, automation is a huge win for you
It gives you the ability to run automation against code that frequently changes to catch regressions in a timely manner
It gives you the ability to run automation in mainstream scenarios to catch regressions in a timely manner
Aids in testing a large test matrix (different languages on different OS platforms).
Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.
Cons of Automation
It costs more to automate.
Writing the test cases and writing or configuring the automate framework you‘re using costs more initially than running the test manually.
Can‘t automate visual references, for example, if you can‘t tell the font color via code or the automation tool, it is a manual test.
Pros of Manual
If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
It allows the tester to perform more ad-hoc (random testing). More bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.
Cons of Manual
1. Running tests manually can be very time consuming
2. Each time there is a new build, the tester must rerun all required tests - which after a while would become very mundane and tiresome.
Other deciding factors
1. What you automate depends on the tools you use. If the tools have any limitations, those tests are manual.
2. Is the return on investment worth automating?
3. Is what you get out of automation worth the cost of setting up and supporting the test cases, the automation framework, and the system that runs the test cases?
Criteria for automating
There are two sets of questions to determine whether automation is right for your test case: Is this test scenario automatable?
1. Yes, and it will cost a little
2. Yes, but it will cost a lot
3. No, it is no possible to automate
How important is this test scenario?
1. I must absolutely test this scenario whenever possible
2. I need to test this scenario regularly
3. I only need to test this scenario once in a while
If you answered #1 to both questions – definitely automate that test
If you answered #1 or #2 to both questions – you should automate that test
If you answered #2 to both questions – you need to consider if it is really worth the investment to automate
What happens if you can‘t automate?
Let‘s say that you have a test that you absolutely need to run whenever possible, but it isn‘t possible to automate. Your options are
Reevaluate – do I really need to run this test this often?
What‘s the cost of doing this test manually?
Look for new testing tools
Consider test hooks
Software Testing Training
Our Software Testing Partner
Software testing institute
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Automated Testing
Automated Testing
Test automation is used to replace or supplement traditional manual software testing with a suite of test programs. Benefits to QA Engineers include increased software quality, repeatable test procedures, and reduced testing costs. Essentially, software automation testing is using a computer system instead of a human to test a software application. Most other forms of software testing require human interaction with the software product under development
Automated Testing is done to:
1. REDUCE TESTING TIME. A typical automated test suite will run in less than 24 hours. For a sophisticated product, manual testing may require dozens of staff months to perform the same testing.
2. CONSISTENT TEST PROCEDURES. With a complex testing process manual testing often yields inconsistent coverage and results depending on the staff and schedule employed. An automated test suite ensures the same scope and process is used repeatedly each time testing is performed
3. REDUCED QA COSTS. Automated testing has an upfront cost to develop, but over the lifetime of a product it will offer substantial net savings. An average automated test suite development is 3-5 times the cost of a complete manual test cycle. Over multiple product releases with multiple cycles per release, this cost is quickly recouped
4. IMPROVED TESTING PRODUCTIVITY. With its much shorter execution time an automated test suite can be run multiple times over the course of a product development cycle
5. IMPROVED PRODUCT QUALITY. Automated testing detects functional and performance issues more efficiently
Points to consider:
1. it's important to define the purpose of taking on a test automation effort. There are several categories of testing tools each with its own purpose. Identifying what you want to automate and where in the testing life cycle will be the first step in developing a test automation strategy. Just wishing that everything should be tested faster is not a practical strategy. You need to be specific
2. Developing a test automation strategy is very important in mapping out what's to be automated, how it's going to be done, how the scripts will be maintained and what the expected costs and benefits will be
3. Many of the testing 'tools' provided by vendors are very sophisticated and use coding 'languages'. Treat the entire process of automating testing as you would any other software development effort. This includes defining what should be automated, (the requirements phase), designing test automation, writing the scripts, testing the scripts,etc. The scripts need to be maintained over the life of the product just as any program would require maintenance
4. The effort of test automation is an investment. More time and resources are needed. The benefit comes from running these automated tests every subsequent release. Therefore, ensuring that the scripts can be easily maintained becomes very important
Software Testing Training
Our Software Testing Partner
Software testing institute
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Test automation is used to replace or supplement traditional manual software testing with a suite of test programs. Benefits to QA Engineers include increased software quality, repeatable test procedures, and reduced testing costs. Essentially, software automation testing is using a computer system instead of a human to test a software application. Most other forms of software testing require human interaction with the software product under development
Automated Testing is done to:
1. REDUCE TESTING TIME. A typical automated test suite will run in less than 24 hours. For a sophisticated product, manual testing may require dozens of staff months to perform the same testing.
2. CONSISTENT TEST PROCEDURES. With a complex testing process manual testing often yields inconsistent coverage and results depending on the staff and schedule employed. An automated test suite ensures the same scope and process is used repeatedly each time testing is performed
3. REDUCED QA COSTS. Automated testing has an upfront cost to develop, but over the lifetime of a product it will offer substantial net savings. An average automated test suite development is 3-5 times the cost of a complete manual test cycle. Over multiple product releases with multiple cycles per release, this cost is quickly recouped
4. IMPROVED TESTING PRODUCTIVITY. With its much shorter execution time an automated test suite can be run multiple times over the course of a product development cycle
5. IMPROVED PRODUCT QUALITY. Automated testing detects functional and performance issues more efficiently
Points to consider:
1. it's important to define the purpose of taking on a test automation effort. There are several categories of testing tools each with its own purpose. Identifying what you want to automate and where in the testing life cycle will be the first step in developing a test automation strategy. Just wishing that everything should be tested faster is not a practical strategy. You need to be specific
2. Developing a test automation strategy is very important in mapping out what's to be automated, how it's going to be done, how the scripts will be maintained and what the expected costs and benefits will be
3. Many of the testing 'tools' provided by vendors are very sophisticated and use coding 'languages'. Treat the entire process of automating testing as you would any other software development effort. This includes defining what should be automated, (the requirements phase), designing test automation, writing the scripts, testing the scripts,etc. The scripts need to be maintained over the life of the product just as any program would require maintenance
4. The effort of test automation is an investment. More time and resources are needed. The benefit comes from running these automated tests every subsequent release. Therefore, ensuring that the scripts can be easily maintained becomes very important
Software Testing Training
Our Software Testing Partner
Software testing institute
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Tuesday, December 16, 2008
White Box Testing Techniques
White Box Testing Techniques
1. Basis Path Testing
a. Flow Graph Notation
b. Cyclomatic Complexity
c. Deriving Test Cases
d. Graph Matrices
2. Control Structure testing
a. Conditions Testing
b. Data Flow Testing
c. Loop Testing
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
1. Basis Path Testing
a. Flow Graph Notation
b. Cyclomatic Complexity
c. Deriving Test Cases
d. Graph Matrices
2. Control Structure testing
a. Conditions Testing
b. Data Flow Testing
c. Loop Testing
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Usability Testing
Usability testing is a special form of testing that looks for bugs not in the functionality of the program, but in the layout and utility of the user interface. This step is often performed on a prototype before the actual system code is written, so it is easy to change if needed. For usability testing, you need to plan:
01. How you will choose the users for the test (what is a representative sample of your real user population?)
02. A set of tasks for the users to perform representing paths through your system showing key functionality
03. A method for getting feedback from the users - surveys, interviews, data collection in the system itself
04. How you will analyze the data you collected in order to make improvements to the user interface
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
01. How you will choose the users for the test (what is a representative sample of your real user population?)
02. A set of tasks for the users to perform representing paths through your system showing key functionality
03. A method for getting feedback from the users - surveys, interviews, data collection in the system itself
04. How you will analyze the data you collected in order to make improvements to the user interface
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Types of Black Box Testing
Acceptance Testing / User Acceptance Testing - UAT Formal tests (often performed by a customer) to determine whether or not a system has satisfied predetermined acceptance criteria. These tests are often used to enable the customer (either internal or external) to determine whether or not to accept a system.
The test procedures that lead to formal 'acceptance' of new or changed systems. User Acceptance Testing is a critical phase of any 'systems' project and requires significant participation by the 'End Users'. To be of real use, an Acceptance Test Plan should be developed in order to plan precisely, and in detail, the means by which 'Acceptance' will be achieved. The final part of the UAT can also include a parallel run to prove the system against the current system.
The User Acceptance Test Plan will vary from system to system but, in general, the testing should be planned in order to provide a realistic and adequate exposure of the system to all reasonably expected events. The testing can be based upon the User Requirements Specification to which the system should conform.
As in any system though, problems will arise and it is important to have determined what will be the expected and required responses from the various parties concerned; including Users; Project Team; Vendors and possibly Consultants / Contractors.
In order to agree what such responses should be, the End Users and the Project Team need to develop and agree a range of 'Severity Levels'. These levels will range from (say) 1 to 6 and will represent the relative severity, in terms of business / commercial impact, of a problem with the system, found during testing. Here is an example which has been used successfully; '1' is the most severe; and '6' has the least impact :-
'Show Stopper' i.e. it is impossible to continue with the testing because of the severity of this error / bug
Critical Problem; testing can continue but we cannot go into production (live) with this problem
Major Problem; testing can continue but live this feature will cause severe disruption to business processes in live operation
Medium Problem; testing can continue and the system is likely to go live with only minimal departure from agreed business processes
Minor Problem ; both testing and live operations may progress. This problem should be corrected, but little or no changes to business processes are envisaged
'Cosmetic' Problem e.g. colours; fonts; pitch size However, if such features are key to the business requirements they will warrant a higher severity level.
The users of the system, in consultation with the executive sponsor of the project, must then agree upon the responsibilities and required actions for each category of problem. For example, you may demand that any problems in severity level 1, receive priority response and that all testing will cease until such level 1 problems are resolved.
Caution. Even where the severity levels and the responses to each have been agreed by all parties; the allocation of a problem into its appropriate severity level can be subjective and open to question. To avoid the risk of lengthy and protracted exchanges over the categorisation of problems; we strongly advised that a range of examples are agreed in advance to ensure that there are no fundamental areas of disagreement; or, or if there are, these will be known in advance and your organisation is forewarned.
Finally, it is crucial to agree the Criteria for Acceptance. Because no system is entirely fault free, it must be agreed between End User and vendor, the maximum number of acceptable 'outstandings' in any particular category. Again, prior consideration of this is advisable.
N.B. In some cases, users may agree to accept ('sign off') the system subject to a range of conditions. These conditions need to be analysed as they may, perhaps unintentionally, seek additional functionality which could be classified as scope creep. In any event, any and all fixes from the software developers, must be subjected to rigorous System Testing and, where appropriate Regression Testing.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
The test procedures that lead to formal 'acceptance' of new or changed systems. User Acceptance Testing is a critical phase of any 'systems' project and requires significant participation by the 'End Users'. To be of real use, an Acceptance Test Plan should be developed in order to plan precisely, and in detail, the means by which 'Acceptance' will be achieved. The final part of the UAT can also include a parallel run to prove the system against the current system.
The User Acceptance Test Plan will vary from system to system but, in general, the testing should be planned in order to provide a realistic and adequate exposure of the system to all reasonably expected events. The testing can be based upon the User Requirements Specification to which the system should conform.
As in any system though, problems will arise and it is important to have determined what will be the expected and required responses from the various parties concerned; including Users; Project Team; Vendors and possibly Consultants / Contractors.
In order to agree what such responses should be, the End Users and the Project Team need to develop and agree a range of 'Severity Levels'. These levels will range from (say) 1 to 6 and will represent the relative severity, in terms of business / commercial impact, of a problem with the system, found during testing. Here is an example which has been used successfully; '1' is the most severe; and '6' has the least impact :-
'Show Stopper' i.e. it is impossible to continue with the testing because of the severity of this error / bug
Critical Problem; testing can continue but we cannot go into production (live) with this problem
Major Problem; testing can continue but live this feature will cause severe disruption to business processes in live operation
Medium Problem; testing can continue and the system is likely to go live with only minimal departure from agreed business processes
Minor Problem ; both testing and live operations may progress. This problem should be corrected, but little or no changes to business processes are envisaged
'Cosmetic' Problem e.g. colours; fonts; pitch size However, if such features are key to the business requirements they will warrant a higher severity level.
The users of the system, in consultation with the executive sponsor of the project, must then agree upon the responsibilities and required actions for each category of problem. For example, you may demand that any problems in severity level 1, receive priority response and that all testing will cease until such level 1 problems are resolved.
Caution. Even where the severity levels and the responses to each have been agreed by all parties; the allocation of a problem into its appropriate severity level can be subjective and open to question. To avoid the risk of lengthy and protracted exchanges over the categorisation of problems; we strongly advised that a range of examples are agreed in advance to ensure that there are no fundamental areas of disagreement; or, or if there are, these will be known in advance and your organisation is forewarned.
Finally, it is crucial to agree the Criteria for Acceptance. Because no system is entirely fault free, it must be agreed between End User and vendor, the maximum number of acceptable 'outstandings' in any particular category. Again, prior consideration of this is advisable.
N.B. In some cases, users may agree to accept ('sign off') the system subject to a range of conditions. These conditions need to be analysed as they may, perhaps unintentionally, seek additional functionality which could be classified as scope creep. In any event, any and all fixes from the software developers, must be subjected to rigorous System Testing and, where appropriate Regression Testing.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Testing Documents
a. User Requirements Specification - URS
The User Requirements Specification is a document produced by, or on behalf of your organisation, which documents the purposes for which a system is required - its functional requirements - usually in order of priority / gradation.
Whilst the URS will not usually probe the technical specification, it will nevertheless outline the expectations and, where essential may provide further detail e.g. the User Interface, say Microsoft Windows®, and the expected hardware platform etc.
The URS is an essential document which outlines precisely what the User (or customer) is expecting from this system. The term User Requirement Specification can also incorporate the functional requirements of the system or may be in a separate document labelled the Functional Requirements Specification - the FRS.
b. Test Plan
A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the 'why' and 'how' of product validation. It should be thorough enough to be useful but not so thorough that no one outside the test group will read it. The following are some of the items that might be included in a test plan, depending on the particular project:
Title
Identification of software including version/release numbers
Revision history of document including authors, dates, approvals
Table of Contents
Purpose of document, intended audience
Objective of testing effort
Software product overview
Relevant related document list, such as requirements, design documents, other test plans, etc.
Relevant standards or legal requirements
Traceability requirements
Relevant naming conventions and identifier conventions
Overall software project organization and personnel/contact-info/responsibilties
Test organization and personnel/contact-info/responsibilities
Assumptions and dependencies
Project risk analysis
Testing priorities and focus
Scope and limitations of testing
Test outline - a decomposition of the test approach by test type, feature, functionality, process, system, module, etc. as applicable
Outline of data input equivalence classes, boundary value analysis, error classes
Test environment - hardware, operating systems, other required software, data configurations, interfaces to other systems
Test environment validity analysis - differences between the test and production systems and their impact on test validity.
Test environment setup and configuration issues
Software migration processes
Software CM processes
Test data setup requirements
Database setup requirements
Outline of system-logging/error-logging/other capabilities, and tools such as screen capture software, that will be used to help describe and report bugs
Discussion of any specialized software or hardware tools that will be used by testers to help track the cause or source of bugs
Test automation - justification and overview Test tools to be used, including versions, patches, etc.
Test script/test code maintenance processes and version control
c. Test Case
01. A test case is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly. A test case should contain particulars such as test case identifier, test case name, objective, test conditions/setup, input data requirements, steps, and expected results.
02. Note that the process of developing test cases can help find problems in the requirements or design of an application, since it requires completely thinking through the operation of the application. For this reason, it's useful to prepare test cases early in the development cycle if possible.
d. Test Script
A test script is basically a script of test cases linked together to walk the critical pathways through an application under test. These scripts are broken down into a series of user scenarios. Each scenario contains specific instructions for the tester to carry out, including the expected results along the way
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
The User Requirements Specification is a document produced by, or on behalf of your organisation, which documents the purposes for which a system is required - its functional requirements - usually in order of priority / gradation.
Whilst the URS will not usually probe the technical specification, it will nevertheless outline the expectations and, where essential may provide further detail e.g. the User Interface, say Microsoft Windows®, and the expected hardware platform etc.
The URS is an essential document which outlines precisely what the User (or customer) is expecting from this system. The term User Requirement Specification can also incorporate the functional requirements of the system or may be in a separate document labelled the Functional Requirements Specification - the FRS.
b. Test Plan
A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the 'why' and 'how' of product validation. It should be thorough enough to be useful but not so thorough that no one outside the test group will read it. The following are some of the items that might be included in a test plan, depending on the particular project:
Title
Identification of software including version/release numbers
Revision history of document including authors, dates, approvals
Table of Contents
Purpose of document, intended audience
Objective of testing effort
Software product overview
Relevant related document list, such as requirements, design documents, other test plans, etc.
Relevant standards or legal requirements
Traceability requirements
Relevant naming conventions and identifier conventions
Overall software project organization and personnel/contact-info/responsibilties
Test organization and personnel/contact-info/responsibilities
Assumptions and dependencies
Project risk analysis
Testing priorities and focus
Scope and limitations of testing
Test outline - a decomposition of the test approach by test type, feature, functionality, process, system, module, etc. as applicable
Outline of data input equivalence classes, boundary value analysis, error classes
Test environment - hardware, operating systems, other required software, data configurations, interfaces to other systems
Test environment validity analysis - differences between the test and production systems and their impact on test validity.
Test environment setup and configuration issues
Software migration processes
Software CM processes
Test data setup requirements
Database setup requirements
Outline of system-logging/error-logging/other capabilities, and tools such as screen capture software, that will be used to help describe and report bugs
Discussion of any specialized software or hardware tools that will be used by testers to help track the cause or source of bugs
Test automation - justification and overview Test tools to be used, including versions, patches, etc.
Test script/test code maintenance processes and version control
c. Test Case
01. A test case is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly. A test case should contain particulars such as test case identifier, test case name, objective, test conditions/setup, input data requirements, steps, and expected results.
02. Note that the process of developing test cases can help find problems in the requirements or design of an application, since it requires completely thinking through the operation of the application. For this reason, it's useful to prepare test cases early in the development cycle if possible.
d. Test Script
A test script is basically a script of test cases linked together to walk the critical pathways through an application under test. These scripts are broken down into a series of user scenarios. Each scenario contains specific instructions for the tester to carry out, including the expected results along the way
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Test Data Categorization Techniques
1. Equivalence Partitioning
Divide the input domain into classes of data for which test cases can be generated.
Attempting to uncover classes of errors.
An equivalence class represents a set of valid or invalid states An input condition is either a specific numeric value, range of values, a set of related values, or a boolean condition.
Equivalence classes can be defined by:
If an input condition specifies a range or a specific value, one valid and two invalid equivalence classes defined.
If an input condition specifies a boolean or a member of a set, one valid and one invalid equivalence classes defined.
Test cases for each input domain data item developed and executed.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Divide the input domain into classes of data for which test cases can be generated.
Attempting to uncover classes of errors.
An equivalence class represents a set of valid or invalid states An input condition is either a specific numeric value, range of values, a set of related values, or a boolean condition.
Equivalence classes can be defined by:
If an input condition specifies a range or a specific value, one valid and two invalid equivalence classes defined.
If an input condition specifies a boolean or a member of a set, one valid and one invalid equivalence classes defined.
Test cases for each input domain data item developed and executed.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Some Testing Terms - part 2
6. SEI
'Software Engineering Institute' at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.
7. CMM
'Capability Maturity Model', now called the CMMI ('Capability Maturity Model Integration'), developed by the SEI. It's a model of 5 levels of process 'maturity' that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors.
Level 1 - characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if any processes in place; successes may not be repeatable.
Level 2 - software project tracking, requirements management, realistic planning, and configuration management processes are in place; successful practices can be repeated.
Level 3 - standard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in place to oversee software processes, and training programs are used to ensure understanding and compliance.
Level 4 - metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.
Level 5 - the focus is on continouous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.
Perspective on CMM ratings: During 1997-2001, 1018 organizations were assessed. Of those, 27% were rated at Level 1, 39% at 2, 23% at 3, 6% at 4, and 5% at 5. (For ratings during the period 1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, and 0.4% at 5.) The median size of organizations was 100 software engineering/maintenance personnel; 32% of organizations were U.S. federal contractors or agencies. For those rated at Level 1, the most problematical key process area was in Software Quality Assurance.
8. ISO
'International Organisation for Standardization' - The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors, and it applies to many kinds of production and manufacturing organizations, not just software. It covers documentation, design, development, production, testing, installation, servicing, and other processes. The full set of standards consists of: (a)Q9001-2000 - Quality Management Systems: Requirements; (b)Q9000-2000 - Quality Management Systems: Fundamentals and Vocabulary; (c)Q9004-2000 - Quality Management Systems: Guidelines for Performance Improvements. To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO certification does not necessarily indicate quality products - it indicates only that documented processes are followed. Also see http://www.iso.ch/ for the latest information. In the U.S. the standards can be purchased via the ASQ web site at http://e-standards.asq.org/
9. IEEE
'Institute of Electrical and Electronics Engineers' - among other things, creates standards such as 'IEEE Standard for Software Test Documentation' (IEEE/ANSI Standard 829), 'IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), 'IEEE Standard for Software Quality Assurance Plans' (IEEE/ANSI Standard 730), and others.
10. ANSI
'American National Standards Institute', the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality).
Other software development/IT management process assessment methods besides CMMI and ISO 9000 include SPICE, Trillium, TickIT, Bootstrap, ITIL, MOF, and CobiT.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
'Software Engineering Institute' at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.
7. CMM
'Capability Maturity Model', now called the CMMI ('Capability Maturity Model Integration'), developed by the SEI. It's a model of 5 levels of process 'maturity' that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors.
Level 1 - characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if any processes in place; successes may not be repeatable.
Level 2 - software project tracking, requirements management, realistic planning, and configuration management processes are in place; successful practices can be repeated.
Level 3 - standard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in place to oversee software processes, and training programs are used to ensure understanding and compliance.
Level 4 - metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.
Level 5 - the focus is on continouous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.
Perspective on CMM ratings: During 1997-2001, 1018 organizations were assessed. Of those, 27% were rated at Level 1, 39% at 2, 23% at 3, 6% at 4, and 5% at 5. (For ratings during the period 1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, and 0.4% at 5.) The median size of organizations was 100 software engineering/maintenance personnel; 32% of organizations were U.S. federal contractors or agencies. For those rated at Level 1, the most problematical key process area was in Software Quality Assurance.
8. ISO
'International Organisation for Standardization' - The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors, and it applies to many kinds of production and manufacturing organizations, not just software. It covers documentation, design, development, production, testing, installation, servicing, and other processes. The full set of standards consists of: (a)Q9001-2000 - Quality Management Systems: Requirements; (b)Q9000-2000 - Quality Management Systems: Fundamentals and Vocabulary; (c)Q9004-2000 - Quality Management Systems: Guidelines for Performance Improvements. To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO certification does not necessarily indicate quality products - it indicates only that documented processes are followed. Also see http://www.iso.ch/ for the latest information. In the U.S. the standards can be purchased via the ASQ web site at http://e-standards.asq.org/
9. IEEE
'Institute of Electrical and Electronics Engineers' - among other things, creates standards such as 'IEEE Standard for Software Test Documentation' (IEEE/ANSI Standard 829), 'IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), 'IEEE Standard for Software Quality Assurance Plans' (IEEE/ANSI Standard 730), and others.
10. ANSI
'American National Standards Institute', the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality).
Other software development/IT management process assessment methods besides CMMI and ISO 9000 include SPICE, Trillium, TickIT, Bootstrap, ITIL, MOF, and CobiT.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Some Testing Terms - part 1
1. Validation:
The comparison between the actual characteristics of something (e.g. a product of a software project and the expected characteristics).Validation is checking that you have built the right system.
2. Verification:
The comparison between the actual characteristics of something (e.g. a product of a software project) and the specified characteristics.Verification is checking that we have built the system right.
3. Configuration Management
Configuration management covers the processes used to control, coordinate, and track: code, requirements, documentation, problems, change requests, designs, tools/compilers/libraries/patches, changes made to them, and who makes the changes.
4. When to stop testing
This can be difficult to determine. Many modern software applications are so complex, and run in such an interdependent environment, that complete testing can never be done. Common factors in deciding when to stop are:
Deadlines (release deadlines, testing deadlines, etc.)
Test cases completed with certain percentage passed
Test budget depleted
Coverage of code/functionality/requirements reaches a specified point
Bug rate falls below a certain level
Beta or alpha testing period ends
5. Risk Analysis/ Identifying Test Cases
Use risk analysis to determine where testing should be focused. Since it's rarely possible to test every possible aspect of an application, every possible combination of events, every dependency, or everything that could go wrong, risk analysis is appropriate to most software development projects. This requires judgement skills, common sense, and experience. (If warranted, formal methods are also available.) Considerations can include:
01 Which functionality is most important to the project's intended purpose?
02 Which functionality is most visible to the user?
03 Which functionality has the largest safety impact?
04 Which functionality has the largest financial impact on users?
05 Which aspects of the application are most important to the customer?
06 Which aspects of the application can be tested early in the development cycle?
07 Which parts of the code are most complex, and thus most subject to errors?
08 Which parts of the application were developed in rush or panic mode?
09 Which aspects of similar/related previous projects caused problems?
10 Which aspects of similar/related previous projects had large maintenance expenses?
11 Which parts of the requirements and design are unclear or poorly thought out?
12 What do the developers think are the highest-risk aspects of the application?
13 What kinds of problems would cause the worst publicity?
14 What kinds of problems would cause the most customer service complaints?
15 What kinds of tests could easily cover multiple functionalities?
16 Which tests will have the best high-risk-coverage to time-required ratio?
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
The comparison between the actual characteristics of something (e.g. a product of a software project and the expected characteristics).Validation is checking that you have built the right system.
2. Verification:
The comparison between the actual characteristics of something (e.g. a product of a software project) and the specified characteristics.Verification is checking that we have built the system right.
3. Configuration Management
Configuration management covers the processes used to control, coordinate, and track: code, requirements, documentation, problems, change requests, designs, tools/compilers/libraries/patches, changes made to them, and who makes the changes.
4. When to stop testing
This can be difficult to determine. Many modern software applications are so complex, and run in such an interdependent environment, that complete testing can never be done. Common factors in deciding when to stop are:
Deadlines (release deadlines, testing deadlines, etc.)
Test cases completed with certain percentage passed
Test budget depleted
Coverage of code/functionality/requirements reaches a specified point
Bug rate falls below a certain level
Beta or alpha testing period ends
5. Risk Analysis/ Identifying Test Cases
Use risk analysis to determine where testing should be focused. Since it's rarely possible to test every possible aspect of an application, every possible combination of events, every dependency, or everything that could go wrong, risk analysis is appropriate to most software development projects. This requires judgement skills, common sense, and experience. (If warranted, formal methods are also available.) Considerations can include:
01 Which functionality is most important to the project's intended purpose?
02 Which functionality is most visible to the user?
03 Which functionality has the largest safety impact?
04 Which functionality has the largest financial impact on users?
05 Which aspects of the application are most important to the customer?
06 Which aspects of the application can be tested early in the development cycle?
07 Which parts of the code are most complex, and thus most subject to errors?
08 Which parts of the application were developed in rush or panic mode?
09 Which aspects of similar/related previous projects caused problems?
10 Which aspects of similar/related previous projects had large maintenance expenses?
11 Which parts of the requirements and design are unclear or poorly thought out?
12 What do the developers think are the highest-risk aspects of the application?
13 What kinds of problems would cause the worst publicity?
14 What kinds of problems would cause the most customer service complaints?
15 What kinds of tests could easily cover multiple functionalities?
16 Which tests will have the best high-risk-coverage to time-required ratio?
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Control Structure testing
a. Conditions Testing
Condition testing aims to exercise all logical conditions in a program module.
Errors in expressions can be due to:
1. Boolean operator error
2. Boolean variable error
3. Boolean parenthesis error
4. Relational operator error
5. Arithmetic expression error
Condition testing methods focus on testing each condition in the program. Strategies proposed include:
Branch testing - execute every branch at least once.
Domain Testing - uses three or four tests for every relational operator.
Branch and relational operator testing - uses condition constraints
Coverage of the constraint set guarantees detection of relational operator errors.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Condition testing aims to exercise all logical conditions in a program module.
Errors in expressions can be due to:
1. Boolean operator error
2. Boolean variable error
3. Boolean parenthesis error
4. Relational operator error
5. Arithmetic expression error
Condition testing methods focus on testing each condition in the program. Strategies proposed include:
Branch testing - execute every branch at least once.
Domain Testing - uses three or four tests for every relational operator.
Branch and relational operator testing - uses condition constraints
Coverage of the constraint set guarantees detection of relational operator errors.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Bugs
Bug
A programming error that causes a program to work poorly, produce incorrect results, or crash. (As an interesting aside, the term "bug" was coined when a real insect damaged one of the circuits of the first electronic digital computer, the ENIAC.)
Isolation
Think about the sequence of events that you just took the software through to eventually get to the problem. Ideally, you started testing by clicking one button, and then saw the problem immediately. More likely, though, you had been testing for a while, possibly for hours. Perhaps the last thing you did is the only thing required to reproduce the bug, or maybe you have to repeat hours of testing. Until you can prove that a simpler scenario is sufficient, you have to assume that every detail of your testing session is relevant. Your task is to rule out as many of those details as you can as not being relevant to the problem.
There are several different things that are subject to simplification. Consider:
Procedures. This is usually what testers focus on – shortening the step-by-step interaction with the system.
Inputs. This is all the data that you feed to the program, such as a command-line argument, a text field in a GUI interface, a file, or database. You want to also reduce this data to the smallest data set that still reproduces the problem.
Configuration. What options have you selected that are different from the default configuration? If you can't reproduce the problem the way the software is configured out of the box, find the few crucial settings that are necessary for the problem to show up.
Platforms. Can you reproduce the problem on all of the operating systems, operating system versions, and hardware combinations that are supported? If not, then you've found an important clue. Also, what about other software that is running, and their versions? Many bugs are not platform-specific, and testing on other platforms can sometimes be difficult, so this area often isn't thoroughly explored.
Other state information. The items above probably don't capture every possible relevant variable. Look for other things that might vary from one system to another and cause the bug to manifest on some systems but not others
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
A programming error that causes a program to work poorly, produce incorrect results, or crash. (As an interesting aside, the term "bug" was coined when a real insect damaged one of the circuits of the first electronic digital computer, the ENIAC.)
Isolation
Think about the sequence of events that you just took the software through to eventually get to the problem. Ideally, you started testing by clicking one button, and then saw the problem immediately. More likely, though, you had been testing for a while, possibly for hours. Perhaps the last thing you did is the only thing required to reproduce the bug, or maybe you have to repeat hours of testing. Until you can prove that a simpler scenario is sufficient, you have to assume that every detail of your testing session is relevant. Your task is to rule out as many of those details as you can as not being relevant to the problem.
There are several different things that are subject to simplification. Consider:
Procedures. This is usually what testers focus on – shortening the step-by-step interaction with the system.
Inputs. This is all the data that you feed to the program, such as a command-line argument, a text field in a GUI interface, a file, or database. You want to also reduce this data to the smallest data set that still reproduces the problem.
Configuration. What options have you selected that are different from the default configuration? If you can't reproduce the problem the way the software is configured out of the box, find the few crucial settings that are necessary for the problem to show up.
Platforms. Can you reproduce the problem on all of the operating systems, operating system versions, and hardware combinations that are supported? If not, then you've found an important clue. Also, what about other software that is running, and their versions? Many bugs are not platform-specific, and testing on other platforms can sometimes be difficult, so this area often isn't thoroughly explored.
Other state information. The items above probably don't capture every possible relevant variable. Look for other things that might vary from one system to another and cause the bug to manifest on some systems but not others
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Basis Path Testing
Basis Path Testing
Aim is to derive a logical complexity measure of a procedural design and use this as a guide for defining a basic set of execution paths.
Test cases which exercise basic set will execute every statement at least once.
a. Flow Graph Notation
Notation for representing control flow
On a flow graph:
1. Arrows called edges represent flow of control
2. Circles called nodes represent one or more actions.
3. Areas bounded by edges and nodes called regions.
4. A predicate node is a node containing a condition
Any procedural design can be translated into a flow graph.Note that compound Boolean expressions at tests generate at least two predicate node and additional arcs.
b. Cyclomatic Complexity
The cyclomatic complexity gives a quantitative measure of the logical complexity.
This value gives the number of independent paths in the basis set, and an upper bound for the number of tests to ensure that each statement is executed at least once.
An independent path is any path through a program that introduces at least one new set of processing statements or a new condition (i.e., a new edge)
Example has:
1. Cyclomatic Complexity of 4. Can be calculated as:
1. Number of regions of flow graph.
2. #Edges - #Nodes + 2
3. #Predicate Nodes + 1
2. Independent Paths:
1. 1, 8
2. 1, 2, 3, 7b, 1, 8
3. 1, 2, 4, 5, 7a, 7b, 1, 8
4. 1, 2, 4, 6, 7a, 7b, 1, 8
Cyclomatic complexity provides upper bound for number of tests required to guarantee coverage of all program statements.
c. Deriving Test Cases
1. Using the design or code, draw the corresponding flow graph.
2. Determine the cyclomatic complexity of the flow graph.
3. Determine a basis set of independent paths.
4. Prepare test cases that will force execution of each path in the basis set.
Note: some paths may only be able to be executed as part of another test.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Aim is to derive a logical complexity measure of a procedural design and use this as a guide for defining a basic set of execution paths.
Test cases which exercise basic set will execute every statement at least once.
a. Flow Graph Notation
Notation for representing control flow
On a flow graph:
1. Arrows called edges represent flow of control
2. Circles called nodes represent one or more actions.
3. Areas bounded by edges and nodes called regions.
4. A predicate node is a node containing a condition
Any procedural design can be translated into a flow graph.Note that compound Boolean expressions at tests generate at least two predicate node and additional arcs.
b. Cyclomatic Complexity
The cyclomatic complexity gives a quantitative measure of the logical complexity.
This value gives the number of independent paths in the basis set, and an upper bound for the number of tests to ensure that each statement is executed at least once.
An independent path is any path through a program that introduces at least one new set of processing statements or a new condition (i.e., a new edge)
Example has:
1. Cyclomatic Complexity of 4. Can be calculated as:
1. Number of regions of flow graph.
2. #Edges - #Nodes + 2
3. #Predicate Nodes + 1
2. Independent Paths:
1. 1, 8
2. 1, 2, 3, 7b, 1, 8
3. 1, 2, 4, 5, 7a, 7b, 1, 8
4. 1, 2, 4, 6, 7a, 7b, 1, 8
Cyclomatic complexity provides upper bound for number of tests required to guarantee coverage of all program statements.
c. Deriving Test Cases
1. Using the design or code, draw the corresponding flow graph.
2. Determine the cyclomatic complexity of the flow graph.
3. Determine a basis set of independent paths.
4. Prepare test cases that will force execution of each path in the basis set.
Note: some paths may only be able to be executed as part of another test.
Software Testing Training
Our Software Testing Partner
Software testing institute
corporate training software testing
For More Visit Site
http://www.qacampus.com
For discussion FORUM
http://www.qacampus.com/forum
Subscribe to:
Posts (Atom)