Introduction

As the most important work output of a test engineer, Bug Reports (or defect reports, issue tickets) are the direct manifestation of a test engineer’s work results and the output that best demonstrates their value and capabilities. However, project stakeholders and even test engineers themselves often underestimate the importance of Bug Reports. The content of bug reports, their submission methods, and handling processes significantly impact project progress, operational efficiency, and even team morale.

There has been relatively little industry discussion and summary regarding Bug Reports. In this article, based on the author’s years of professional observations, we hope to provide some summaries and suggestions on the elements of Bug reports and how professional test engineers should handle them. We aim to spark discussion, and welcome corrections for any oversights.

Main contents include:

• What is a Bug, and what is a Bug Report
• What information do development or project teams actually want to get from Bug reports
• Common mistakes made by testing novices when submitting bugs
• Bug reporting and storytelling
• How professional test engineers submit high-quality bugs

What is a Bug? The History of Bugs

The word “Bug” in English originally meant insect or beetle, and is now used in the IT industry to refer to defects in software. This history is quite interesting:

As early as 1878, the great inventor Edison used the word “bug” in a letter to a friend to refer to faults (or potential causes of faults) in electronic circuits. However, its true entry into computer industry history came on September 9, 1947.

At that time, the first programmers were developing the Mark II computer at Harvard University. [Grace Murray Hopper][1] found a moth near a relay contact while investigating a problem, killed it with her logbook, and marked it as “the first actual case of finding a bug.” Since then, the word “bug” has been used to refer to defects or faults in computers. Correspondingly, “debug” is used to refer to the act of removing faults.

The image below is the first actual bug preserved in the Smithsonian Museum:

Among the first programmers in history, the only woman was the “mother of the bug”:

In any case, the word “Bug” now refers to defects or faults in software or programs. However, for project teams, it’s recommended not to simply define Bug as a fault, as the word fault has a negative connotation. Viewing Bugs as defects or issues, emphasizing room for improvement has a more positive guiding effect on project teams.

What is a Bug Report?

The word “report” in English can be both a verb and a noun. So Bug report can refer to both the action of finding and reporting a problem, and the report content submitted after finding a problem. Here we generally view it as a problem report (noun), i.e., an issue ticket, so what exactly is an issue ticket?

Testing guru Cem Kaner once pointed out:

The purpose of testing is not to find bugs. The purpose is to get bugs fixed.

This view actually points out a misconception many testers have - thinking that testing work is just about finding as many bugs as possible. In fact, for the team, no matter how many bugs are found, they ultimately need to be resolved to demonstrate the value of finding these bugs.

From discovery to resolution, the issue ticket is the main medium, being both the output of testers’ work and the input for developers to solve problems, its importance is self-evident.

Additionally, while there are various ways to inform developers of discovered problems - verbal notification, actual demonstration, email, IM chat, etc., can all serve the purpose of notification. However, in modern software development processes, there are usually dedicated systems for defect tracking, such as JIRA, Zentao, Bugfree, QC (HP Quality Center), etc., which can handle defect submission and tracking. Through these specialized systems, not only can they serve the purpose of recording and feedback, but they can also better reflect the overall quality status and development progress of the project through statistics and analysis.

Therefore:

• An issue ticket is the output after testers discover defects during testing activities

• An issue ticket is the input for developers to resolve defects

• An issue ticket is the communication medium between testers and developers regarding bugs

• An issue ticket is also an indicator for project teams to evaluate current development status and product maturity.

The Role and Importance of Issue Tickets

Many test engineers have probably experienced the following feedback or complaints after submitting issue tickets:

• What exactly are you trying to say with this issue ticket?

• I can’t reproduce this problem on my end

• Why do you consider this a defect?

• Normal people don’t use software the way described in the issue ticket, only someone with mental issues would use it that way

• This is indeed a problem, but the cost of fixing it is too high, and there are alternative ways to work around it

• You might think this is a problem, but actual users won’t find it problematic

• While what you submitted is indeed an issue, it has minimal impact on users. Why don’t we see many issue tickets for those parts that have a greater impact on users?

  ……

Testers certainly don’t want to receive such feedback. But these issues also indirectly reflect that when it comes to bugs, developers or project teams actually focus more on how to resolve bugs (including judging whether they need to be resolved), and they hope to get the following information from issue tickets:

• What problem was discovered?

  • What is the problem phenomenon?
  • Does it always occur?
  • What was the background and context when discovering this problem?
  • Besides the visible phenomenon, what else happened in the system?
  • Are there screenshots, logs, screen recordings?

• Why is it a problem?

  • What impact does this problem have?
  • What happens if it’s not resolved?
  • What’s the probability of occurrence?
  • Is it duplicated with other issues?

• What operations were performed that exposed this problem?

  • What was the input?
  • What test path and test data were used?
  • What about the test environment and software version?

Therefore, as testers, we shouldn’t view issue tickets as simple problem records. They are important communication media between testing and development. Good issue tickets can greatly reduce communication costs between both parties, both greatly reducing developers’ workload in solving problems and reducing testers’ workload in clarifying and re-verifying. For the project, the quality of issue ticket submission affects the team’s productivity and delivery speed, which cannot be underestimated.

Elements of Issue Tickets

Understanding the role of issue tickets, the elements that should be included in them become quite clear, generally including the following aspects:

• Title for preliminary judgment

  The title should clearly summarize the problem and the module where it occurs, making it convenient for developers to make an initial judgment

• Context and necessary related information

  The contextual background when the problem occurs, related information such as software version, test environment, configuration status, resource status, etc.

• Accurate and reasonable detailed description

  Detailed description of the steps to discover the problem, operation expectations, actual conditions that occurred. Problem occurrence frequency, test inputs, data, files used, etc.

• Supplementary information to help resolve the problem

  Supplementary information that can help developers speed up problem resolution, such as screenshots, screen recordings, operation logs, system logs, etc.

• Auxiliary information for project decision-making

  As the first judge of bugs, testers should provide suggestions on the severity of the problem and resolution priority. Initially assign developers to resolve the problem, and provide related information identifiers for statistics and tracking.

Common Mistakes by Testing Novices

Combining the elements of issue tickets, we can initially summarize some common mistakes made by testing novices or non-professional testers when submitting bugs:

• Unclear titles

  Titles that are too simple make it difficult to make preliminary judgments about the problem. For example, simply writing “system crash” is too vague, and overly simple titles can easily become similar, causing inconvenience for focused review and evaluation of issues.

  Or titles that are too complex. Long titles can lead to reading fatigue and lack focus. For example, there’s no need to write detailed operation steps in the title.

• Unrealistic, deviating from requirement scenarios

  Sometimes testing novices make assumptions based on personal preferences and raise issues. “I think” and “it feels not good” are high-frequency words in such novice issues.

• Not actually problems

  Testing novices often submit issue tickets that aren’t actually bugs due to misunderstandings of system implementation principles or lack of skill level.

  Also, problems caused by incorrect testing methods or environment configuration by testing novices themselves.

• Insufficient description information

  Typically, through the description information in the issue ticket, developers cannot confirm the problem phenomenon or reproduce it themselves. Testing novices often think their job is done once they find a problem, without providing sufficient information to developers, such as screenshots and logs.

• Too much irrelevant information, messy formatting

  Sometimes novice issue tickets contain too much redundant noisy information, making it difficult to see the key points. For example, pasting tens of thousands of lines of logs into the issue ticket indiscriminately.

  Or completely lacking formatting, with a lot of information mixed together messily, making it extremely difficult to read.

• Too many issues

  Issue tickets should be focused, but sometimes testing novices include many different issues in the same ticket. This is very unfavorable for issue tracking and focusing.

• Only looking at phenomena without considering essence

  Sometimes simple problem phenomena are often manifestations or clues of serious problems. Novices often overlook important risks, ignoring problems hidden beneath seemingly minor traces, easily missing bugs lurking beneath phenomena. For example, some unexpected prompt messages are often manifestations of backend calculation and statistical accuracy bugs.

Bug Reporting and Storytelling

Coming back to the act of submitting bugs, essentially, this is a communication process. The issue ticket carries the role of this communication medium. Through the issue ticket, testers describe the problems they see and discover during testing, and developers understand the corresponding information by reading the issue ticket and resolve the problems accordingly.

So submitting bugs is actually like writing a short article or telling a story, where testers are the authors and developers are the readers.

When we submit issue tickets, it’s actually a form of writing. The 5W requirements for writing also apply:

• What - What exactly is the problem?

• Why - Why raise this issue? Why is it a problem?

• Who - Who is the subject of the problem? Who are the related parties?

• Where - Where did the problem occur?

• When - When and at what frequency does it occur?

When submitting bugs, thinking about these 5Ws, combined with the issue ticket elements mentioned above, generally helps avoid many mistakes that testing novices commonly make when submitting issue tickets.

The Art of Bug Reporting

Above we discussed the elements of issue tickets and some details to note when submitting bugs, but why do we say that submitting bugs is actually an artistic work?

Let’s look at some examples of issue tickets:

Suppose there’s a product management system where users experience a crash when adding new products under a specific product category (fruits)

Issue Ticket Example 1:

1
2
3

User adds product, crash occurs

As shown in the image (crash screen screenshot)

This issue ticket basically describes the problem phenomenon through the provided screenshot. However, for developers, there’s too little information, and they likely won’t be able to reproduce the problem immediately (fruits category). The information contained in the issue ticket is too simple, making it an extremely poor issue ticket example.

Issue Ticket Example 2:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

User experiences crash when adding product

Operation steps:
1. Open App
2. User logs into system, enters correct username and password and logs in
3. Enter product management page
4. Select category: fruits
5. Click new product button
6. Enter product information: *********
7. Click submit button
Expected result: Product created successfully
Actual result: Crash occurs
Device model used: Xiaomi 8
Operating system version: Android 8.0
Software version V1.1
Attachment: Crash screenshot

This issue ticket is a common submission format by many testing novices, containing very detailed operation steps and related auxiliary information. But this is also a very poor issue ticket. First, such step descriptions are unnecessary - developers don’t need to be told about basic operations like opening the app, logging in, and submitting in the issue ticket. Second, while there’s a lot of auxiliary information, it’s mostly irrelevant to the problem cause and doesn’t help developers locate the issue, instead introducing some distracting information. Such issue tickets fail to demonstrate the professional value of testers and don’t help with quick problem resolution.

Issue Ticket Example 3:

1
2
3
4
5
6
7
8
9

User experiences crash when adding product under fruits category

Operation steps:
1. Select fruits category, create product, crash occurs
2. Select non-fruit category, create product, product created successfully

Verified on models: Xiaomi 8, Huawei Mate 7 etc. show consistent phenomenon, should be unrelated to model, operating system, version
Attachments: Error screenshot, system crash logcat logs
Impact: Users cannot add products in fruits category, directly affects user experience. Needs high priority resolution

This issue ticket can be considered a relatively qualified one. It clearly indicates the scenario where the problem occurs and has preliminarily eliminated some irrelevant information. Developers can focus on product creation under the fruits category for problem analysis. It also points out the problem’s impact on the software and resolution priority suggestions, making the project team clearly aware of the problem’s impact.

Issue Ticket Example 4:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

User experiences null pointer crash when adding product under fruits category

Operation steps:
1. Select fruits category, create product, crash occurs
2. Select non-fruit category, create product, product created successfully
3. Edit existing products under fruits category, edit successful

Backend database insert operation fails due to missing field for fruits category, update operation normal.

Verified on models: Xiaomi 8, Huawei Mate 7 etc. show consistent phenomenon, should be unrelated to model, operating system, version
Attachments: Error screenshot, system crash logcat logs
Impact: Users cannot add products in fruits category, directly affects user experience. Needs high priority resolution

This issue ticket goes further than the previous one, preliminarily analyzing the problem cause - during new operations, a key field information is missing, causing the crash. Developers can now very clearly proceed with fixing the problem.

Issue Ticket Example 5:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

User experiences null pointer crash when adding product under fruits category. Product import operation with old template has the same issue.

Operation steps:
1. Select fruits category, create product, crash occurs
2. Select non-fruit category, create product, product created successfully
3. Edit existing products under fruits category, edit successful

Backend database insert operation fails due to missing field for fruits category, update operation normal.

In the product management import function, new template fruit product import successful, using old template (missing new field) to import fruit data also fails. Needs to be fixed together.

Verified on models: Xiaomi 8, Huawei Mate 7 etc. show consistent phenomenon, should be unrelated to model, operating system, version
Attachments: Error screenshot, system crash logcat logs, new and old template import logs
Impact: Users cannot add products in fruits category, directly affects user experience. Needs high priority resolution

This issue ticket, building on the previous example, demonstrates the tester’s professional capability by reasoning and verifying that the import function with similar logic might have the same issue based on their experience.

Through this simple case above, we can see that different ways of submitting issue tickets can have very different effects on problem resolution.

• Too little information makes it difficult for developers to determine the root cause of the bug, multiplying the debugging workload.

• Too much information causes developers to spend extra effort analyzing redundant information, even interfering with debugging and resolution direction, similarly causing significant waste of work effort.

• The more clearly and accurately the root cause of the problem is reflected, the more efficiently developers can solve the problem.

• The tester’s experience and judgment reasoning ability can be well demonstrated through issue tickets.

So why do we say submitting bugs is an artistic work? Issue tickets should neither be too simple nor too complex, and should be able to convey information as accurately and comprehensively as possible to developers - this is truly an artistic endeavor.

Conclusion

In summary, submitting bugs is a very important activity in the software development process, and the quality of issue tickets directly affects team efficiency and output. As professional testers, mastering the art of submitting bugs means that issue tickets are not simple problem records, but rather communication bridges between testing and development, supporting developers in efficiently fixing bugs.

Welcome to follow and comment, you can also follow the WeChat public account: 秋草说测试