This article is a series of articles (as listed below) in a distributed, J2EE-based project (as listed below).
Part 1: Project introduction; high-level plan Part 2: Risk management; demand management
Part 3: Model Creating and Access Control; Demand Analysis Section 4: Use Case Refining; Equipment Report; Tools and Techniques Part 5: System Architecture and Design Part 6: Detailed Design; Early Development; Early Engineering; Unit testing Part 7: Continue development; early construction; demonstration Part 8: Unit test strategy; functional testing; GUI test script
Part 9: System construction and testing; defect tracking; product delivery
Part 10: Project completion; conclusion; future work
In this article, we are a software company Lookoff Technologies Incorporated, our customers AudioPhile Speaker Design, Inc. (ASDI), which hires us to achieve their initial IT needs. For more detailed information, see Part 1.
This article is a part 8 of this series of arts, which discusses in detail in the topic introduced in this series of tests in this series. In the article on Section 6, we have seen the bottleneck of the usage and performance of the memory in early development we started using Rational Purify and Rational Quantify. At the same time, we also discussed many details in our early unit testing. This article will describe the progress of these work and review the ability of our automated testing of the Rational Test Tool to reduce the cost of the test, mainly the usage of Rational Purecoverage and Rational Robot. At this stage of the project, we mainly pay attention to functional testing (including GUI test), although we have also engaged in some early load testing.
Note that the Rational Unified Process (RUP) term used herein reflects two different dimensions of the test: "Unit Test" is based on the development phase of the software to be tested - at this time test is for minimal testing Software units - and "Function Test" and "Load Test" are for specific test targets, whether it is in the development phase of the test software. Most of the contents of the unit tests discussed herein can also be applied to test work in our later development phase (for example, incorporating different components or subsystems during integration tests).
Part 8 Snapshots in Part 8 Demo:
Rational Purecoverage - The coverage of the code during unit test (the number of code executed) Rational Robot - Execute Recording and Playback Test Scripts Rational Administrator - Used to create a project, use it with Rational Robot, association Test script with projects Rational TestManager - Organizational and management tests, and view the results of the tests being created or updated:
Robot test script - created for automated testing
In the development of the unit test, we found that our unit testing has exceeded our development work because we are building a very complex user interaction. Unit tests always require a lot of work, and we may not have sufficient budget and time to perform a large number of unit testing work. We found our GUI test, including the process of monotonous manual test, especially slowing our speed.
Although the first stage of the ASDI project is just a conceptual verification, we also have difficult time to endure the bugs in the system. It is very difficult to find a balance between fast demonstrations with rugged, reliable software and frequent integration. We define the scope of the first phase of the project so that we must conduct more than one technical demonstration; customer expectation is a Beta version of the system that can be used. Here, we will take a look at our unit testing, we can choose (or not) using automated capabilities, and we use Rational Purecoverage to determine our testing thoroughness.
Test range unit test range - what, how many times and time test - is a variable foundation, including:
The complexity of the software is characteristic and the interface is tested to the tolerance of the defect in the software for the entire product.
In terms of code testing, we did not intend to test 100% code coverage in our software. Complete coverage is very expensive and it is difficult to achieve because we have to manually check our code to determine which line of code is not tested. Small changes to unit testing will require us to retrieve the code to ensure that the full coverage is maintained. However, we have to thank Rational Purecoverage, which we can easily achieve close 100% code coverage (due to the reasons for project budgets and scope, a small amount of unsurified residue is allowed). PURECOVERAGE greatly simplifies tasks for checking code coverage, which makes the code that identifies which part of the code is or not tested very simple.
As we mentioned in this series of Part 6, our unit test work is almost the same as the core software develops itself. We most developers start writing unit tests in their software units that have been tested. They like to test the internal components of the test software when they produce them in their minds.
The execution of unit tests is always constructed in the system; it will be clearly introduced into a build version to be eliminated by unit testing. It is unacceptable. Unit tests are always executed before the code is distributed. In addition, developers run their unit test in a standardized basis to ensure that their regular changes will not undermine the software. Our method is not as radical as some software methods - such as Extreme Programming (XP), the development of unit test in XP is usually the development of the code, but we use the unit test as an important step in an important software development. .
Automated capabilities This will certainly have some questions that test, especially what kind of test should be automated. Developers who write "Background" (non-GUI) code are lucky, they can write automated tests, including Java drivers, code stubs, and scripts. However, as we mentioned earlier, the test of GUI developers is more difficult.
For our non-GUI test, we observed the habit of unit testing associated with each class; we write the driver code access class and report success or fail. We tried to use test frameworks that can automatically generate, manage, and run tests for us, such as those recommended by XP, but they will produce mixed results. Although theory is good, these frameworks (including, such as JUnit) are not mature enough for our project purposes. However, our developers are very confident if they have time to improve and improve these tools, these test frameworks are very valuable to our unit test. We may use more of these test frames in the future project. In our ASDI project, the use of XP may be too risky; instead, we are more willing to accept the concepts and ideas of XP in control and risk reduction. You can find a lot of content about XP thinking on the xprogramming.com website. We didn't use Rational's test tool to test non-GUI's code, because we think they can't reach a level of bottom. For automated testing our GUI-driven code, Rational's test tool is really excellent. User interface test is very manual and interactive, and requires a lot of content confirmation; Rational Test Tools makes us more simple and easy to make these tests in a consistent and fast way.
The code coverage is based on Rational Purecoverage, during our unit test execution, we can significantly reduce our time to determine the code coverage - That is, which methods or code rows are executed or miss it. This tool enables us to observe the code coverage test we provide, and then determine whether to test the code by manually checking.
In addition to supporting C or C applications, PurecoveRage can also support Java programs, and can be configured and running in the same way as Quantify (in this series of Part 6). This tool enables us to browse our unit test; Code override test in two ways: Use the Coverage Browser or the Coverage FileView way to specify the file.
Using the Coverage Browser method (Figure 1), we can see the statistical summary of the code coverage test of each file (and directory) included in the unit test. These statistics include all the number of calls, the number of methods accessed or omissions, and the number of code lines that are accessed or missing.
Figure 1: Cover Browser (click to enlarge)
We can double-click a method in summary to enter the Coverage fileView, only which code lines are displayed in that method or miss it. As is shown in the example shown in Figure 2, the missing code line is highlighted in red (blue identity to the code line).
Figure 2: Coverage FileView (Click Here to Enlarge)
Function Test Function Test verifies whether the software is working correctly based on the needs of the defined in the case. To ensure we tested all the needs, we use Rational Robot and Rational TestManager when designing and organizing our functional test. (A discussion on the use of Robot's Function Test Policy can be found in the Robot's User Guide.) Robot enables the test script to be recorded, and then play back for automated test. Use Robot to create two types of test scripts:
The GUI script records your user interface action. When playback, just like a window of the login and control program of the operation application. Some client tests - for example, an ActiveX control embedded in our web app - you need to use these scripts. Because the GUI test has caused a special problem, this type of functional test will be discussed in detail in the next part. VU Scripts detects your application when you use it to track what network information is sent and accepted on the package level. When playback, these scripts do not have to log in to the application, you can re-run the test, which makes them faster than the GUI script. We use the VU script for our Command Gateway's non-GUI function test and B2B load test. We start to create our functional test collection as early as possible, because these tests are very valuable for engineering teams. We are sometimes difficult to implement before the code is distributed to check, which may be difficult to implement, may be because some parts of the code and some components are completed more time than other code and components, or because of integration And the test (I & T) team is too slow to complete the test script in time. In one case, when the user interface is still changing, the test script has been completed; although some rework is necessary, the appearance of changes is fairly easy to integrate.
Once we have a strong functional test set, the retest system will be very easy. This takes a lot of rewards when you need to retest system small changes in later iterations, and make sure that changes will not affect other code parts. We map our function test scripts to the usage case so that we can immediately know which test should be run when the need to be modified when it is associated to a given case.
GUI Test When the developer of the non-GUI code is very efficient to carry out their unit testing, the developer of JSP / servlet will be difficult to ensure the progress of the time. Stinging them is the process of testing the GUI domain, which includes repeated execution below:
Establish test data in the database. Log in to the application via a web browser. Navigate the page tested (a form is to be populated). Fill in one or more domains on the page and rely on specific tests. Click the appropriate button to submit the form. test result. Return to step 4 using different tests (different domain values or different domains) until all reasonable combinations are tested.
Some GUI tests are especially annoying, including the same entrance, fill in the lengthy values in several domains. You can use cut and paste, but for the team, repeating these tests is still very trivial work. Some pages have 50 different tests must be performed, and each test includes many steps.
Although we did not tend to use Rational's test tools, we think that our difficulties we have invested in in-depth study of the Rational Test Tools. We decided to use Rational Robot to distinguish most of our GUI's tests. We have adjusted these tests in the following ways:
Developers drafted a unit test for each test. The primary integration and test team members accept unit test instructions and create test scripts based on the current software build version. The unit test script is configurable to manage the change of the test script can be tracked. Regularly, a developer gives an integrated and test team to perform unit test scripts, and provide a report that indicates success or failure based on unit test instructions.
This method works very well. It is not only in the early stage of the development process of integration and testing teams, but also gives them to more work, but also gives them a chance to understand the entire software system by revealing unit test instructions. In addition, we have prepared other functional test scripts for GUI testing using Robot.
When Rational Robot starts, it requires a selection that exists (using Rational Administrator), and the test script will be associated with this item. In our example, the project's positioning must be accessible on the web so that we can share the database of the project across the team. We created an ASDI project in Administrator, as shown in Figure 3, and when the Robot prompt us when we select a project, we chose this project. Figure 3: Create a project using Rational Administrator
An example of a verification test As a very simple example of a GUI test, we need to confirm that the client's authentication is correct for our PartSearch.jsp page, this page performs JavaScript code to verify that they domain. Each ASDI condition, some numbers are input to the entire page, for example, must be a large number than 0; if a user enters an invalid number, JavaScript code will report an error and will not send bad Data to the server. Our JavaScript code followed the sample code for form validation, we wrote our own verification code in the Formchek.js file. Appropriate use of this code, a negative input will pop up the window shown in Figure 4.
Figure 4: Appropriate PartSearch.jsp page verification
Creating a test script We created a test script in the Robot to test the authentication of the digital domain shown appropriate. First Robot prompts us to provide a name for the script (Figure 5).
Figure 5: Recording a new GUI script
After clicking OK, we have provided a recording controller (Figure 6), and then we use our window-based applications when Robot recording our actions. In this example, we launched Internet Explorer 5, browsing on PartSearch.jsp page, enter a bad number (-123456), click the Submit button on the page.
Figure 6: Robot recording controller
Robot records our actions in a GUI script (Figure 7). This is a very simple example; our actual test will include more complex test scripts. We don't have to create hundreds of separate scripts, but try to design a large script for each screen interface. In Robot's article, there are very detailed information on test design, we can get a lot of reference in the contents recommended in the Robot document.
Figure 7: Simple Robot GUI script (click to enlarge)
By using easy-to-read and easy to edit scripts, we don't have to repeat the test script for tests that only have a small difference. Sometimes we can edit the test script without re-running the test, or we just re-record a small part of a test, then paste it into the existing script.
Run Test To click on the test, we simplify on the Robot's recording controller (or select Playback on the File menu) Click the "Play" button. The script is like playing back our test operation like we initially performing tests, and when the script is complete, Rational TestManager is started to summarize our test results. For the simple example above, the results shown in Figure 8 show that the result of the test script run is the test being passed.
Figure 8: TestManager Report Robot test pass (click to enlarge)
Suppose we want to test the error handling ability of the code, such as the bad number no longer captured - that is, no window indicating the error, our application will transfer the entire data to the servlet to perform the query. When we re-run this GUI test by playback script, Robot will capture this problem and write the results into the report (Figure 9), including displaying this test failure. Figure 9: TestManager reports the failure of the Robot test (click to enlarge)
Summary When we start ASDI project, we expect to use Rational's analysis and design tools rather than Rational's test tools. To the first phase of the project, we were very surprised and very happy to see we automated and improve our testing process by using Rational test tools. These tools have a solid learning curve, but once we master the usage of each of our tools, our integration and test team will greatly improve. In some cases, individual developers use tools like Rational Purify, Quantify, and Purecoverage to supplement their unit testing. Other tools, such as Rational Robot, need to integrate with the test team with higher skills and put more attention.
Planning future teams must still bear the tasks of integration and testing on the system level. Most unit testing works to developers, but we need a more comprehensive and formal test subsystem and the entire system. This means that we need formal build versions, the ultimate build documentation and important attention in integration and testing. The load test of a particular purpose is just a small part of our test work, but it is very important in the entire system. Fortunately, our test collection has completed 95%, and we are still a little more than planned time, so we can have some extra energy to pay attention to the system's test to ensure high quality first-stage system. deliver.
Main risk
Our risk list is already very short. Customers in the evolutionary system are not surprising. Our technical risk is very small, thank you very much for the good progress brought about by the engineering team.
Now we must develop our code to develop final processing, implement system tests, identify any defects in the system, and give customers a system in accordance with the scheduled delivery phase. When the team approaches the end of the major phase, it is difficult to pack all the details. If we want to avoid the plan, we need a small amount of defects, and can quickly correct any problems found in the test.
About the author Steven Franklin has a very broad background in software design, architecture, and engineering process, which is often used in large, distributed information management and control systems. He started using Rational tools in 1997, his main interest in XML, J2EE, wireless and software engineering technology. You can contact Steven via Steve@sfranklin.net.
