What is Defect Management?

You are starting a project and know some defects will probably be created. But you also know that the person who found the defect will tell a developer, and the developer will fix it right away, test it to make sure it works right this time, and put it back into the application code, and all will be good with the world. Right?

Well, not so much. Pretty much every piece of that puzzle has problems. Yes, the testers will find defects. That much we have as a given. However, how do we know that defect exists? How do we know a developer was told about it? How do we know they are working on it and not something they feel is more important? How do we remember to check to see if it was fixed, and when the developer does get around to fixing it, how do we know it gets back into the application code and how do we know that it now works, and did not break something else in the meanwhile? Simple answer - without a Defect Management process, we don't.

Just what is Defect Management? We see it as a small (but very important) piece of the full Quality Assurance umbrella. This tail end of the QA process controls defects from when they are found to when they are closed, sets up the Defect Management environment before testing begins, manages and tracks the defects (and resources working on them), and reporting on defect metrics throughout the process. And who is in charge of doing all of this? The Defect Manager. This individual, whether part time or full time, needs to be the one main resource for making sure the process works smoothly from start to finish.

Without the Defect Management processes in place, we mainly risk defect not being fixed, and that defect escaping into the wild of production. All of the pieces of Defect Management are put in place as protection against that happening.

So, rather than assuming that defects will be handled well, we need to assure it. We do this with planning the execution of Defect Management, and executing the plan. Steps that you will want to ensure are executed include:

1. Setting up defect your repository
2. Publicizing and training the teams on the defect process
3. Ensuring the defects are entered correctly into the repository
4. Prioritizing and assigning defects during triage
5. Tracking and accurately reporting the defect population lifecycle
6. Verifying that each defect is retested in a timely manner and added to the next build
7. Identifying key trends hidden in the defects

Each of these needs some explanation, and several could generate a blog by themselves, so let's take a look at them at a high level only for this blog:

1. Setting up defect your repository

The first step, before any defects need to be logged, is to set up (or verify/modify) the tool you are using for a defect repository, where defects will be logged, tracked, assigned, reported from, and closed. This is one of the steps that could use a blog all for itself, but let's hit the highlights here.

1. What does the defect form look like?
   • a. Do you have all of the needed fields on the form for assigning, tracking and reporting the defect?
   • b. Are they laid out cleanly, in an intuitive flow?
   • c. Are the correct fields made required, or read only, or alternately not required or editable?
   • d. Are drop-down fields fully and correctly filled with the list of possible values?
2. Do you have (or should you have) different access rights to fields depending on the role of the user accessing the defect form? For instance, you may not want your developer to be able to close out a defect, but only a tester or administrator.

2. Publicizing and training the teams on the defect process

Now, you have everything set up correctly in your repository tool. But, does anyone know how to use it? You will need to set up one or more Archival documents (PowerPoint, Word, etc.) for all future audiences, and one or more Training documents (most likely PowerPoint or something like that for live classes to users). I say one or more since there may be one for how to enter Test Cases for execution, how to execute those test cases, and then, as pertains to this blog, how to write defect found while either executing test cases or during exploratory testing (fooling around with the application). The first set of documents, the descriptive ones, will be the most complete, having verbiage and images about every aspect of using the tool, and creating defects. Those are for reference by any new or existing team member. The second set, the Training documents, are more for classroom training of the team member doing the testing, step by step on how to log and update defects.

3. Ensuring the defects are entered correctly into the repository

So, your testers are ready to start testing and logging defects. But, it is up to the Defect Manager, as part of the Defect Management process, to ensure, correct, and "re-train" the users on how best to enter defects into the system, if necessary. These could be corrections to selections (modifying the testing cycle or environment, entered incorrectly by the tester, for instance). But, ultimately, it is part of the process to ensure that the fields are correctly filled out.

4. Prioritizing and assigning defects during triage

Triage? Where did this come from? Simply put, the triage meetings should be held as many times as twice a day to keep the workflow of defect refactoring flowing smoothly, depending on the number and criticality of the defects. Triage meetings need to be in place for two reasons. To determine that:

1. New defects are filled out and prioritized correctly, assigned to the correct resource as a next step, and that key people are informed of critical and high
2. Existing defects are reviewed and it is verified that they is movement on them, that they are on their way to being fixed, not forgotten in a corner somewhere. A review of critical and high severity defects needs to be part of the triage meetings to ensure action is being taken by them, and if nothing has been happening, to contact the appropriate resource to find out why.

The appropriate people need to be in each of these triage meetings. These would include the Defect Manager, owners (department heads) of the application area(s) being tested, and key development resources in those same areas.

5. Tracking and accurately reporting the defect population lifecycle

Metrics describing the overall status of the defect refactoring effort may not directly address the fixing of any given defect, but is invaluable to making sure that attention and effort is being directed properly, that resources are being focused appropriately. As the saying goes, you cannot fix what you do not measure. And measurement is key to determining if and where things might be going wrong.

In order to create metrics useful to Sr. Management to make decisions, the proper fields need to be tracked in the defect form. These include, but should not be limited to:

• Defect Summary or Title
• Status or State
• Severity
• Defect Type
• Defect location
• Date Created
• Date Closed
• Closed Reason
• Root cause
• Detected by
• Assigned to
• Cycle/release detected in
• Environment encountered in
• Business process, if tracked
• Owner, either a resource or a department

Once your process includes the tracking and proper updating of all of these fields, a variety of metrics can be generated and distributed to allow Sr. Management to act if needed to address issues.

These metrics may include the following and many more, depending on what the management desires:

1. Current number of New, Assigned, Fix in Progress, Ready to Test, Closed defects
2. Simple number of open and closed defects by severity
3. Velocity of Opened defects (how many opened per day, how many resolved per day, number currently open)
4. Duration of defect lifetime (how many days, on average, defects were open before resolved, broken by severity to compare against SLAs for refactoring, if that is being tracked)
5. Defect Status by Team/Application/Functional Area by Status/Severity
6. Defects by Defect Type/Root Cause
7. Count of defects over time by Team/Application/Functional Area

And the list could go on and on. Note that each of those listed above are suggestions, but be creative. TALK to the Sr. Management to see what they need to see, and how they need to see the data. Tables, charts? What colors in the charts makes sense to them? Bar, column, pie, line or Doughnut?

6. Verifying that each defect is retested in a timely manner and added to the next build

Once the defect state shows that it is ready to retest, ideally the original tester who found and logged the defect should retest it to verify the application now works correctly and can pass the test cases that caught the defect, and test around the defect to verify that the fix did not accidentally break something else. Note that the title of this section includes the words "in a timely manner". A good suggestion is that every morning, when the testers arrive, have grabbed their coffee or tea, and are ready to get started testing again, the first open up the defect repository and look for any defects that are assigned to them and are ready to test, and test those first, before they are consumed by the rest of the testing effort. In that manner, defects that need retesting are not neglected and sit for too long without being addressed.

7. Identifying key trends hidden in the defects

Metrics sent to the Sr. Management are one thing. The management may be focused on specific numbers on which they want to act, such as number of Critical severity defects. But, they only know what they know - that is, they are only aware of the numbers they have asked for.

It is part of the process of Defect Management and the job of the Defect Manager to scour the data in the defect repository and find trends that point to potential issues, and communicate those to the Sr. Management. For instance, if the Defect Manager notices that there happens to be a long delay once the defect gets to the developers' desks, or once they are ready for retest, and that they sit in that state for a long while, and this is not something that the Sr. Management is currently tracking, it is worth a mention that this is a bottleneck and may need to be addressed. If the defect reason or root cause of defects is not being reported up, but the Defect Manager finds that many defects are being logged for poor data, or perhaps coding errors, but due to one developer in particular. These are not trends that are normally going to come out in standard metrics requested by Sr. Management, but the Defect Manager has, as part of their responsibilities within the Defect Management process, to look at the data in every way that they can imagine to pull this type of information out, and present it to the appropriate people if it looks to be an issue to the success of the project.

So, there it is - Defect Management in a nutshell. Get things ready, make sure everyone is trained, ensure proper logging, prioritization and assignment, track and report appropriately, ensure defects are resolved quickly, and identify trends that could impact the project timeline and success. As I had mentioned earlier, most of these key pieces in the above diagram could have a blog all on their own, as they each hold a very important place in the overall success of Defect Management and therefore of the project as a whole.