Lecture 3 - Test Cases and Test Suites -> Quick Intro -> Test Case Structure -> Results Of The Test Case Execution -> Useful Attributes Of The Test Case -> Data-Driven Test Cases -> Maintainability Of Test Cases -> The Number Of Expected Results Inside One Test Case -> Bad Test Case Practices -> Test Suites -> States Of A Test Case -> The Importance Of Creativity -> Three Important Factors That Affect Test Cases -> Checklists -> Lecture Recap -> Questions & Exercises1. The test case is a piece of test documentation prepared and used by a tester to check for one or more expected results.
2. The test case steps (procedure) serve as an instruction of how to reach an actual result (output). Excessive detailing of steps causes difficulty in test case maintenance, while excessive abstraction causes difficulty in test case execution.
3. The steps for repetitive scenarios can be moved to a separate helper document inside QA Knowledge Base, so in the test case we simply refer to that document: e.g., “Do Checkout (see QA KB).”
4. The test case execution ends with a PASS or FAIL result. A FAIL result is desired, because in that case we have a bug (provided that the test case reflects what’s really expected).
5. The test case execution is considered incomplete if the tester couldn’t execute all the steps: e.g., situations where the test case execution is blocked because of a bug in the middle of the execution.
6. A test case should be independent from all other test cases; i.e., there should be no reliance on test cases from its own and/or any other test suites. A good way to check the independence of test cases inside a given test suite is to change the order in which the test cases are executed (i.e., if the test cases are truly independent, the order of execution doesn’t matter).
7. The following test case attributes are extremely useful:
– A unique ID is unique not only within a test suite, but also among all test cases existing in the company.
– A priority is a value from 1 to 4 (inclusively) which reflects the importance of a test case. The practical consequence of having differences in priorities: A P1 test case is more important than a P2 test case, and if we have time to execute only one of the two, we would execute the P1 test case.
– An IDEA – a written description (in human language) of a test case idea.
– The SETUP AND ADDITIONAL INFO gives the test case executor helpful information for test case execution. This section is commonly used to store data, account information, SQL statements, and other pieces of information which help to execute test cases and make them more maintainable.
– The revision history helps us to know history behind test case modifications.
8. Test case maintainability is about the ease and comfort with which we are able to modify a test case. Test case maintainability is one of the most important formal aspects to keep in mind during test case generation/modification.
9. Each test case checks only one testing idea, BUT two or more expected results are totally legitimate if we need to perform several verifications for that testing idea.
10. Below are examples of bad practices during test case generation/modification:
– Dependency between test cases;
– Poor description of the steps;
– Poor description of the IDEA and/or expected result.
11. Test cases are combined into test suites. As a rule, the whole test suite is placed into one file; e.g., a MS Word document.
12. As a rule, test suites include test cases which check
– the same area of the software (e.g., “Checkout”);
– items from the same spec.
13. It’s a good practice to create a new test suite for each new spec. Even if the new test cases are written to test the same functional area (e.g., Credit Cards), it’s better to create a temporary test suite with new test cases and then move survivors into the permanent test suite after first execution.
14. In the majority of situations, test cases are written after a simple reading of the spec, talking to developers, and taking other actions to understand what the future software is supposed to do. In other words, your familiarity with any future software AND its testing is purely theoretical. Thus, test case modification during the first execution is always a reality. Fresh test cases are not supposed to be perfect. After all, everyone knows what an abyss can exist between a theory and a practice.
15. It’s imperative that during test case generation/modification we always think about the testers who will execute test cases after us.
16. Just like humans, test cases have the following states: “Created,” “Modified,” and “Retired.” Unlike humans, test cases don’t die. We want to keep them alive for a number of reasons; for instance, as a memory. So, if a certain test case is no longer applicable, we don’t brutally delete it, but simply cut and paste it into the file with the same name as test suite file, but located in a special directory (“retired_testcases”). If the whole test suite cannot be used anymore, then we just move it into the same “retired_testcases” directory.
17. Be creative!!! And remember that
– Importance of the project
– Complexity of the project
– Stage of the project
affect formal and actual side of test cases.
18. In this lecture we’ve covered only the formal side of test cases. It’s important to understand that the gist of a test case’s existence is to help us find bug (-s). Bug finding techniques will be explained in great detail later on. Next ->
Lecture 3 - Test Cases and Test Suites -> Quick Intro -> Test Case Structure -> Results Of The Test Case Execution -> Useful Attributes Of The Test Case -> Data-Driven Test Cases -> Maintainability Of Test Cases -> The Number Of Expected Results Inside One Test Case -> Bad Test Case Practices -> Test Suites -> States Of A Test Case -> The Importance Of Creativity -> Three Important Factors That Affect Test Cases -> Checklists -> Lecture Recap -> Questions & Exercises
