Estimating Bugs — Yes or No?
The things you need to consider deciding whether it will help or hurt your team
About the Author
Kiryl Baranoshnik is an experienced Agile and Lean practitioner. He works as an Agile Coach at a large bank in the Netherlands and shares his knowledge publicly as an Agile trainer.
Kiryl Baranoshnik is an experienced Agile and Lean practitioner. He works as an Agile Coach at a large bank in the Netherlands and shares his knowledge publicly as an Agile trainer.
In my previous post, I tried to answer the question of what to do with the bugs found during an iteration. After that, a few people have asked whether estimating bugs is a good idea and if it is how to do it. Obviously, there are only two options and each of them has their pros and cons. Before drawing conclusions, let's take a look at both. Note that here I assume we are talking about a team that works in sprints and does sprint plannings.
Not Estimating Bugs
- It helps to focus on the delivery of value. The logic here is that when a team calculates velocity they only count new feature development — the only part of the work that provides end-user value. Bugs are considered merely one of many factors that lower velocity. This (supposedly) helps the team to be more rigorous about not letting bugs escape because no one wants their velocity to drop to zero. The argument against this is that velocity is a poor metric for value. The only true thing it measures is the team's capacity which does not necessarily translate to value.
- Long-term planning becomes more accurate. During long-term planning, you rarely estimate bugs because at the moment of planning bugs are yet unknown. If your velocity only includes new features and your forecast only includes new features, you can simply divide forecast by velocity to get your estimated date of completion. The counter-argument here is that new features are continuously added to the backlog at any moment in development, many of them after the long-term planning. Thus, they are as unknown as bugs and not estimating bugs doesn't make it easier. Instead, tracking work items by type (e.g. planned, unplanned, bugs) will provide you with enough data to adjust any kind of long-term forecast.
- Bugs are hard or impossible to estimate. Oftentimes bugs are much harder to estimate. This is especially true for old bugs or bugs found in production because the team has already lost the context. Oftentimes the team itself hasn't written the buggy code that they need to fix. One can answer to this that new stories might be just as vague and hard to estimate as old bugs. Fresh bugs, on the other hand, can be estimated relatively easily since the team still has the necessary context and there're not so many dependencies in the code yet.
Estimating Bugs
- It's much easier for a team to plan sprints. In the end, velocity is a capacity metric and its purpose is to help a team assess their own capacity for the next sprint. This task becomes much simpler if you include all kinds of work in the calculation.
To Estimate or Not to Estimate?
Generally, I find that teams feel much more comfortable with estimating bugs and the idea of only measuring "new stuff" doesn't provide such a great psychological effect as it's supposed to. I used to have a strong opinion about this in the past and insisted on not estimating bugs. Over time I've come to realize that what's more important is keeping your estimation as transparent, clear, and consistent as possible. So, generally, I estimate bugs with my teams. In the end, forecasting is a manager/Agile Coach's jobs and if it complicates things for you but makes it easier for the team — that should be your preferred course of action.
How to Estimate
Considering the thought that you need to be consistent with your estimations, the answer becomes pretty obvious. You want to estimate them in the same way you estimate everything else. That means if you estimate in story points using Planning Poker, you should do Planning Poker on bugs too. If you estimate with Magic Estimation (which I find more helpful), bugs should also be on your estimation board.