There is no "gold standard" method for game testing, and most methodologies are developed in-house, by individual video game developers and publishers. And, methodologies are refined from time to time and may differ for different types of games (for example, the methodology to test a MMORPG will be different from the testing for a casual game). But outlined below is a typical methodology of game testing.
Identification
Finding a bug and actually recognizing it is a defect (as opposed to a feature).
Defect typing
Assigning a specific type to the bug. For example, missing art, game crash, misspelled text, etc.
Analysis
This step of the process involves the party responsible for the bug, such as an artist, programmer or game designer. Often tools are used in helping tracking down the flaw.
Debugging tools
The type of debugging tool used depends on the type of defect. Many defects don't require any specialized tools at all, and just require the user to play the game and look for the reported flaw. Some flaws, however, such as those located in the actual code, require the use of specialized tools, such as an IDE, which help finding erroneous sections of code. Art errors, for example, may require use of a 3D modeler, Photoshop or an in-house tool.
Recording tools
A testing department may use any variety of tools to record discovered defects. Most will employ a database which can be queried, which helps in locating identical bugs and tracking the correction of bugs. There are many software packages that are developed just for this type of work. However, a specialized package is not always necessary, depending on the scale of the game.
Reporting
Reporting a bug to the responsible party may be done in person or automatically with tools (mentioned above, software for recording bugs). Sometimes, individual game testers do not report bugs themselves, but hand them off to a lead tester who then determines and informs the responsible parties.
Tracking
After a bug is reported, it is tracked, either through an automated system (software) or by a person assigned to the task (such as the reporting tester or the test lead).
Defect tracking systems
In engineering, defect tracking is the process of finding defects in a product, (by inspection, testing, or recording feedback from customers), and making new versions of the product that fix the defects. Defect tracking is important in software engineering as complex software systems typically have tens or hundreds or thousands of defects: managing, evaluating and prioritizing these defects is a difficult task: defect tracking systems are computer database systems that store defects and help people to manage them.
Verification
After a defect has been reported as corrected, it must be checked. Often, the original reporter of the defect verifies the fix, but, if it is sufficiently documented, it can be verified by another party.
Regression testing
As game development progresses, it is necessary to go back and re-check features, art and text (content) that previously was error-free to ensure they still are. Though tedious, this type of testing is extremely important since late and supposedly benign changes to code and other content can have produce disastrous side effects that may make some features inoperable or can destroy the game as a whole.
Regression testing is any type of software testing which seeks to uncover regression bugs. Regression bugs occur whenever software functionality that previously worked as desired stops working or no longer works in the same way that was previously planned. Typically regression bugs occur as an unintended consequence of program changes.
Common methods of regression testing include re-running previously run tests and checking whether previously fixed faults have re-emerged.
Experience has shown that as software is developed, this kind of reemergence of faults is quite common. Sometimes it occurs because a fix gets lost through poor revision control practices (or simple human error in revision control), but just as often a fix for a problem will be "fragile" - i.e. if some other change is made to the program, the fix no longer works. Finally, it has often been the case that when some feature is redesigned, the same mistakes will be made in the redesign that were made in the original implementation of the feature.
Therefore, in most software development situations it is considered good practice that when a bug is located and fixed, a test that exposes the bug is recorded and regularly retested after subsequent changes to the program. Although this may be done through manual testing procedures using programming techniques, it is often done using automated testing tools. Such a 'test suite' contains software tools that allow the testing environment to execute all the regression test cases automatically; some projects even set up automated systems to automatically re-run all regression tests at specified intervals and report any regressions. Common strategies are to run such a system after every successful compile (for small projects), every night, or once a week.
Regression testing is an integral part of the extreme programming software development methodology. In this methodology, design documents are replaced by extensive, repeatable, and automated testing of the entire software package at every stage in the software development cycle.
Test group organization
The organization of any test group differs from organization to organization, but a typical one may include the individuals below.
Game producer: responsible for the game as a whole
Test Manager: responsible for all testing for game, and game testers
Test Lead: responsible for all testing, sometimes identical to Test Manager
Test Engineers: often called game testers, the people who carry out the testing
Test Analyst: analyzes and develops test plans
No comments:
Post a Comment