Lecture 6 - Classifying the Most Common Types of Testing -> Quick Intro -> By Knowledge Of The Internals Of The Software -> By The Object Of Testing -> By Time Of Test Execution– Functional testing
– UI testing
– Usability testing
– Localization testing
– Load/performance testing
– Security testing
– Compatibility testing
FUNCTIONAL TESTING
Functional testing is required to find logical bugs in the Web site functionalities. Test Case with Credit Card is an example of functional testing. We’ll cover black box testing techniques for functional testing in a separate lecture.
UI TESTING
UI testing is required to find bugs in the user interface of the Web site.
You should understand the difference between:
1) UI testing
2) Testing using UI
In first case, you test UI.
Example
ShareLane -> Here is typical test case for UI testing:
1. Go to https://www.sharelane.com/cgi-bin/main.py.
2. Click the “Sign up” link.
3. Check the number of characters that can be typed into the “ZIP code” text field.
Expected result: 5
In second case, you test whatever you need and simply use UI as a means to bring the software to a certain state.
Example
ShareLane -> Here is a typical test case where we just use UI to test logic of the code:
1. Go to https://www.sharelane.com/cgi-bin/main.py.
2. Click the “Sign up” link.
3. Type “2008” into “ZIP code” text field and press the “Continue” button.
Expected result: error message that ZIP code must have 5 digits.
We will cover some topics about UI testing, but our main focus will be functional testing. Why?
– Functional testing is much more important and much more difficult than UI testing.
– Most software testing activities are about functional testing.
USABILITY TESTING
Usability testing is the evaluation of the user experience when he uses our software. Sometimes usability testing is regarded as part of UI testing, and sometimes it’s not.
Example 1: Each of us have seen “genius” Web site interfaces that are like an intricate puzzle when you attempt to perform some basic action (like finding the store locator page on some pizza Web site).
Example 2: One of the reasons why folks prefer Facebook to MySpace is that Facebook has a neat, conservative look, while MySpace users often abuse their ability to change the look of their Web pages by producing ugly monsters that hurt your eyes by flashing pictures and white text on a yellow background.
BTW
In the blessed nineties when VCs were joyfully pouring millions into the bank accounts of freshly baked start-ups, it wasn’t unusual for start-ups employees to go out onto the streets of San Francisco and beg strangers to do usability testing for 50+ bucks an hour. Those were the days, my friends…
Feel free to talk to the PM if you think that your Web site’s usability sucks.
LOCALIZATION TESTING
Localization testing is required to find bugs in the adaptation of our software for users from different countries. For example, if our Web site was created for an English-speaking audience and we want to localize it for a Japanese-speaking audience, we’ll have to check if Kanji symbols can be used to create a username.
LOAD/PERFORMANCE TESTING
Load/performance testing is a set of testing techniques required
– To check the response time of our Web site or its components
– In case of a problem with response time, to find out what the problem is
The response time of a Web site is usually about how fast a user sees the next page after he does some kind of interaction – e.g., submit a registration form.
The most common technique is to incrementally increase the number of users until some concrete benchmark (e.g., 4 seconds of response time) is broken. That way, we can approximately predict the maximum number of users that can use our site within a reasonable response time. In many cases, this testing is done to discover a bottleneck. A bottleneck is a part of the software or hardware that slows down the response time. In some cases, the bottleneck is just an SQL statement that consumes too much of the server’s memory, so the resolution of this performance problem can be as easy as modifying that statement, or dividing it into two or more separate statements.
The most common way to deal with problems concerning response time is to add extra machines to the production pool.
SECURITY TESTING
Several years ago, one of my friends who had just arrived in the U.S. from Russia refused to use the Internet because she “was afraid of hookers.” Every time she said this, we’d laugh like mad, because what she really meant was that she was afraid of “hackers,” i.e., cyber criminals.
Cyber crime is a HUGE problem. Because of it, individuals and corporations lose billions of dollars every year. In his interview in the book Founders at Work, PayPal co-founder and CTO Max Levchin said, “2000 was basically the year of fraud, where we were just losing more and more money every month. At one point we were losing over $10 million per month in fraud. It was crazy.“
Security testing refers to testing the protection against security breaches. In many cases, that testing takes form of simulated hacker attacks.
COMPATIBILITY TESTING
Compatibility testing is about checking the compatibility of our software with the hardware and/or software on a user’s computer. For example, some projects might have special requirements for a user’s video cards (hardware), so compatibility testing might be dedicated to checking how compatible the software is when used with different video cards. With Web-based applications, in the majority of cases, compatibility testing takes the form of cross-browser and/or cross-platform testing.
Example
Many years ago, during one project we found a bug where a certain version of OS “A” would restart if the image was uploaded to our Web site using a certain version of Web browser “B”.
– When we tried to upload the image with a different version of “A” and the same version of “B” everything was OK.
– When we tried to upload the image with a different version of “B” and the same version of “A” everything was OK.
Therefore, the problem was that a specific OS+Web browser combo was not compatible with our Web site.
There are three conclusions here:
1. Compatibility testing involves a variety of third party hardware and/or software with which our software must be tested.
2. Compatibility problems are a reality, and a user might have a really bad experience because of those problems.
3. For a Web project, it’s a good idea to have a test lab with computers that have different OS and Web browsers.
As you know, one of the sources of expected results is statistics.
– You can get statistics about the worldwide usage of OS and Web browsers if you Google it (e.g., “Web browser usage statistics”), or you can check out the links in the Statistics section under Downloads on qatutor.com.
– You can also ask your company’s IT person to give you some stats about the OS and Web browsers of your company’s users.
BTW
The IT person gets those stats from Apache logs. Apache logs have that info because the Web browser sends it as a part of the “user-agent” section of the HTTP header. You can Google the phrase “my user-agent” and discover how Web sites “see” your OS and Web browser.
Many developers and testers use Firefox, because it provides many useful plug-ins and generic features. But during testing, testers should follow the rule, “Eat your dog’s food,” and use Web browsers preferred by users. Google “most popular web browsers” for the current statistical info.
In my experience, compatibility problems mostly occur when a Web site uses some cutting-edge scripting technology (e.g., AJAX) that might be treated differently by different Web browsers. Next ->
Lecture 6 - Classifying the Most Common Types of Testing -> Quick Intro -> By Knowledge Of The Internals Of The Software -> By The Object Of Testing -> By Time Of Test Execution
