What to automate, when to automate, or even whether one really needs automation are crucial decisions which the testing (or development) team must make. TecStar’s staff are experts in this area. Selecting the correct features of the product for automation largely determines the success of the automation effort and when mistakes are made in the crucial first steps of automation planning, the results can be disastrous. Many a company has given up on Test Automation entirely simply because the wrong choices were made from the very beginning and the results were less than impressive.
If you have experienced this yourself, we sympathize, but we can also prove to you that Test Automation can, and is, worth the initial cost if done correctly from the beginning. Test Automation can literally save companies hundreds of thousands of dollars when done right.
Some software testing tasks, such as extensive low-level interface regression testing, can be laborious and time consuming to do manually. In addition, a manual approach might not always be effective in finding certain classes of defects. Test automation offers a possibility to perform these types of testing effectively. Once tests have been automated, they can be run quickly and repeatedly. Many times, this can be a cost effective method for regression testing of software products that have a long maintenance life. Even minor patches over the lifetime of the application can cause features to break which were working at an earlier point in time.
We employ or can place experts in Quick Test Pro (QTP), Selenium, QFTest, Perl, etc.
In software engineering, performance testing is in general testing performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.
Load testing is the simplest form of performance testing. A load test is usually conducted to understand the behavior of the system under a specific expected load. This load can be the expected concurrent number of users on the application performing a specific number of transactions within the set duration. This test will give out the response times of all the important business critical transactions. If the database, application server, etc. are also monitored, then this simple test can itself point towards any bottlenecks in the application software.
Stress testing is normally used to understand the upper limits of capacity within the system. This kind of test is done to determine the system’s robustness in terms of extreme load and helps application administrators to determine if the system will perform sufficiently if the current load goes well above the expected maximum.
Soak testing, also known as endurance testing, is usually done to determine if the system can sustain the continuous expected load. During soak tests, memory utilization is monitored to detect potential leaks. Also important, but often overlooked is performance degradation. That is, to ensure that the throughput and/or response times after some long period of sustained activity are as good or better than at the beginning of the test. It essentially involves applying a significant load to a system for an extended, significant period of time. The goal is to discover how the system behaves under sustained use.
Spike testing is done by suddenly increasing the number of or load generated by, users by a very large amount and observing the behaviour of the system. The goal is to determine whether performance will suffer, the system will fail, or it will be able to handle dramatic changes in load.
Rather than testing for performance from the perspective of load, tests are created to determine the effects of configuration changes to the system’s components on the system’s performance and behavior. A common example would be experimenting with different methods of load-balancing.
Isolation testing is not unique to performance testing but involves repeating a test execution that resulted in a system problem. Often used to isolate and confirm the fault domain.
A key step in the process of software engineering is testing the software for correct behavior prior to release to end users.
For small scale engineering efforts (including prototypes), exploratory testing may be sufficient. With this informal approach, the tester does not follow any rigorous testing procedure, but rather explores the user interface of the application using as many of its features as possible, using information gained in prior tests to intuitively derive additional tests. The success of exploratory manual testing relies heavily on the domain expertise of the tester, because a lack of knowledge will lead to incompleteness in testing. One of the key advantages of an informal approach is to gain an intuitive insight to how it feels to use the application.
Large scale engineering projects that rely on manual software testing follow a more rigorous methodology in order to maximize the number of defects that can be found. A systematic approach focuses on predetermined test cases and generally involves the following steps.
Choose a high level test plan where a general methodology is chosen, and resources such as people, computers, and software licenses are identified and acquired.
Write detailed test cases, identifying clear and concise steps to be taken by the tester, with expected outcomes.
Assign the test cases to testers, who manually follow the steps and record the results.
Author a test report, detailing the findings of the testers. The report is used by managers to determine whether the software can be released, and if not, it is used by engineers to identify and correct the problems.
A rigorous test case based approach is often traditional for large software engineering projects that follow a Waterfall model. However, at least one recent study did not show a dramatic difference in defect detection efficiency between exploratory testing and test case based testing.
Testing can be through black-, white- or grey-box testing. In white-box testing the tester is concerned with the execution of the statements through the source code. In black-box testing the software is run to check for the defects and is less concerned with how the processing of the input is done. Black-box testers do not have access to the source code. Grey-box testing is concerned with running the software while having an understanding of the source code and algorithms.
Static and dynamic testing approach may also be used. Dynamic testing involves running the software. Static testing includes verifying requirements, syntax of code and any other activities that do not include actually running the code of the program.
Testing can be further divided into functional and non-functional testing. In functional testing the tester would check the calculations, any link on the page, or any other field which on given input, output may be expected. Non-functional testing includes testing performance, compatibility and fitness of the system under test, its security and usability among other things.