Why I Stopped Using Planning Poker
The simple alternative to Planning Poker that deals with relative estimations in a proper way
About the Author
Kiryl Baranoshnik is an experienced Agile and Lean practitioner. He works as an Agile Coach at a large bank in the Netherlands and shares his knowledge publicly as an Agile trainer.
Kiryl Baranoshnik is an experienced Agile and Lean practitioner. He works as an Agile Coach at a large bank in the Netherlands and shares his knowledge publicly as an Agile trainer.
Estimations is a hot topic in software development. There are articles written, talks given, and books published. When to estimate, how to estimate, and whether to estimate in the first place — all these are not easy questions but I'm going to let you explore them on your own. Let's assume though, you've weighed all the pros and cons and decided to estimate in story points. Chance are, you are looking at Planning Poker. It is probably the most popular estimation technique in the Agile community. I had played numerous sessions with my teams before it gradually started falling out of my favor. And I had reasons for that. But before we get into detail about why you shouldn't use Planning Poker, let's take a look at why you actually might want to use it.
What I Like About Planning Poker
Planning Poker was created by James Grenning in 2002 and seems to be a simplified version of another popular estimation method — Wideband Deplhi (although Grenning himself claims he would have forgotten Wideband Delphi by that time [1]). There were two problems he was trying to solve. First, estimates were taking too much time. Second, existing techniques failed to involve the entire team [2]. Planning Poker wasn't very successful in solving the first problem, as you will see later, but it did well in solving the second one.
Incentivizing Discussion
One of the key elements of Planning Poker is conversation. Participants are expected to discuss the items until a consensus is reached in the estimate. A session facilitator usually makes sure the discussions happen if the group members turn out to be too inert to do this on their own. Sometimes it leads to valuable discoveries.
Protecting From External Influence
It seems to be a side effect of Planning Poker but it's exactly what Wideband Delphi was aiming at. There is some scientific evidence demonstrating that in their estimates people tend to gravitate towards the number they've been exposed to before the estimation started. This happens even if the number is totally random or incongruous. This effect is a cognitive bias and is called anchoring [3]. So, whenever you involve more people in an estimation process, you don't want them to know each other's estimates lest they are not anchored. Another reason for anonymous estimation is that participants tend to go along with the most authoritative opinion in the room. You want to avoid that too. Even simpler than that, people maybe just too lazy to make up their own opinion. In any case, forcing each participant's estimate on their own effectively removes the effect.
Incentivizing Discussion
One of the key elements of Planning Poker is conversation. Participants are expected to discuss the items until a consensus is reached in the estimate. A session facilitator usually makes sure the discussions happen if the group members turn out to be too inert to do this on their own. Sometimes it leads to valuable discoveries.
Protecting From External Influence
It seems to be a side effect of Planning Poker but it's exactly what Wideband Delphi was aiming at. There is some scientific evidence demonstrating that in their estimates people tend to gravitate towards the number they've been exposed to before the estimation started. This happens even if the number is totally random or incongruous. This effect is a cognitive bias and is called anchoring [3]. So, whenever you involve more people in an estimation process, you don't want them to know each other's estimates lest they are not anchored. Another reason for anonymous estimation is that participants tend to go along with the most authoritative opinion in the room. You want to avoid that too. Even simpler than that, people maybe just too lazy to make up their own opinion. In any case, forcing each participant's estimate on their own effectively removes the effect.
What I Don't Like About Planning Poker
At the same time, there are a lot of things that don't work in Planning Poker as well as you would expect them to.
Faster, Still not Fast Enough
While Planning Poker seemingly has achieved some success in speeding up the estimation process, it could do much better in this field. During the Poker sessions a lot of time is still wasted on figuring out how exactly to play the game, especially in inexperienced groups. Turns out, the process is not entirely intuitive and people need to be thoroughly instructed before they fully understand the rules and the dynamics of the game. What's worse, it doesn't seem to improve a lot over time. After a team gets the hang of the rules numerous discussion keep popping up about how exactly the rules should be interpreted. Should we go for another round? Why don't we just take the bigger number? No, you can't just average out, we need to come to a consensus… And it goes on, and on, and on.
In defense of Planning Poker, Grenning in his original paper suggested not to spend too much time on trying to reach consensus. If you can't reach it quickly, don't sweat it. [4] So, with the right approach, the issue can and should be alleviated.
Faster, Still not Fast Enough
While Planning Poker seemingly has achieved some success in speeding up the estimation process, it could do much better in this field. During the Poker sessions a lot of time is still wasted on figuring out how exactly to play the game, especially in inexperienced groups. Turns out, the process is not entirely intuitive and people need to be thoroughly instructed before they fully understand the rules and the dynamics of the game. What's worse, it doesn't seem to improve a lot over time. After a team gets the hang of the rules numerous discussion keep popping up about how exactly the rules should be interpreted. Should we go for another round? Why don't we just take the bigger number? No, you can't just average out, we need to come to a consensus… And it goes on, and on, and on.
In defense of Planning Poker, Grenning in his original paper suggested not to spend too much time on trying to reach consensus. If you can't reach it quickly, don't sweat it. [4] So, with the right approach, the issue can and should be alleviated.
Not Truly Relative
But Planning Poker has a much more serious flaw: it actually makes it harder to use story points properly. In Planning Poker items are estimated one by one and a group is supposed to compare each item to a baseline item (typically a '1'). It's exactly where lies the main problem. In their heads, participants have to figure out how many times bigger the item is and it looks like the human brain is pretty bad with this kind of operation. It tries to simplify the task by creating an image of each size on the scale and unconsciously translates it to time intervals (e.g. 2–3 days for size '2'). While the entire idea of relative estimations is to compare items against each other, people start comparing them to fixed time intervals. This is not much different from estimating in hours or days. After all, hours and days are fixed time intervals. Planning Poker, designed as a relative estimation technique, turns into absolute estimations without you noticing. Have you ever seen during a poker session somebody saying 'it feels more like a 3 to me'? This is a dead giveaway the effect is taking place.
There's a way to get around this problem by triangulating with two or more values used as baselines [5]. It helps, but unlikely removes the problem entirely. There is another method that essentially does the same thing, but does it naturally and is faster and easier to understand.
But Planning Poker has a much more serious flaw: it actually makes it harder to use story points properly. In Planning Poker items are estimated one by one and a group is supposed to compare each item to a baseline item (typically a '1'). It's exactly where lies the main problem. In their heads, participants have to figure out how many times bigger the item is and it looks like the human brain is pretty bad with this kind of operation. It tries to simplify the task by creating an image of each size on the scale and unconsciously translates it to time intervals (e.g. 2–3 days for size '2'). While the entire idea of relative estimations is to compare items against each other, people start comparing them to fixed time intervals. This is not much different from estimating in hours or days. After all, hours and days are fixed time intervals. Planning Poker, designed as a relative estimation technique, turns into absolute estimations without you noticing. Have you ever seen during a poker session somebody saying 'it feels more like a 3 to me'? This is a dead giveaway the effect is taking place.
There's a way to get around this problem by triangulating with two or more values used as baselines [5]. It helps, but unlikely removes the problem entirely. There is another method that essentially does the same thing, but does it naturally and is faster and easier to understand.
Magic Estimation as a Fast and Effective Alternative
So, is there really a better alternative to Planning Poker? I believe there is. It's a family of estimation techniques that share some common characteristics:
- they are visual;
- the estimation is done by moving cards around on a white board;
- the goal is to align the cards in groups left to right by size from small to large.
You may have heard about them under various names: Silent Grouping, Magic Estimation, Affinity Estimation, Swimlanes Sizing. I will call them Magic Estimation (indeed, it feels like magic the first time you experience it). The differences in these methods are minor and which one exactly to use is a matter of taste.
So, what Magic Estimation can offer that Planning Poker can't?
Simple
The rules of the technique are extremely simple. You basically only need to draw the estimation board. Once people see the board, it takes a couple of minutes to explain the gist of the game.
So, what Magic Estimation can offer that Planning Poker can't?
Simple
The rules of the technique are extremely simple. You basically only need to draw the estimation board. Once people see the board, it takes a couple of minutes to explain the gist of the game.
Lightning Fast (With a Caveat)
In all versions of this technique, communication is limited in some way to speed up the exercise. But even in fully unconstrained versions conversations are triggered mostly when there's a disagreement between players. Procedural overhead is very little and no discussion occurs if it's not needed.
Gains in speed come with a caveat, though. You can't expect participant to estimate something they have no clue about, which means they still have to spend time discussing the items they estimate. It would be a lie to say that you can shrink a one-hour Planning Poker session down to 5 minutes, because in Planning Poker discussions are an integral part of the game and in Magic Estimation you need to hold these discussions beforehand as a part of preparation. However, there's still much less overhead since the conversation format is not mandated. I've seen real teams doing two-hour sessions to discuss three months worth of work followed by a 5-minutes estimation exercise. Imagine estimating several dozen items using Planning Poker.
One may argue that while Magic Estimation saves you a lot of time, you might be losing in accuracy as opposed to Planning Poker. It's a valid concern but here's the paradox. Conventional wisdom has it that estimating large backlogs with Planning Poker is slow as hell and thus impractical, so nobody does this anyway. Surprisingly, estimating iteration backlogs with Planning Poker is no more practical. On the scale of two weeks the error margin is a few user stories at the very worst. When so little is at stake, there's no rational reason for perfect estimation accuracy. Instead, it's better to turn your focus and energy to breaking work down into smaller shippable pieces and making the team deliver truly working software at the end of each sprint — that is, actively learning and reducing risks.
Prepackaged with Relative Sizing
The part that I love about Magic Estimation the most is its visual nature. It makes it almost impossible to avoid estimating relative. You are literally forced to compare items against each other by having to place them on the board either left or right. That's the best way I've found so far to quickly make people understand relative estimations.
In all versions of this technique, communication is limited in some way to speed up the exercise. But even in fully unconstrained versions conversations are triggered mostly when there's a disagreement between players. Procedural overhead is very little and no discussion occurs if it's not needed.
Gains in speed come with a caveat, though. You can't expect participant to estimate something they have no clue about, which means they still have to spend time discussing the items they estimate. It would be a lie to say that you can shrink a one-hour Planning Poker session down to 5 minutes, because in Planning Poker discussions are an integral part of the game and in Magic Estimation you need to hold these discussions beforehand as a part of preparation. However, there's still much less overhead since the conversation format is not mandated. I've seen real teams doing two-hour sessions to discuss three months worth of work followed by a 5-minutes estimation exercise. Imagine estimating several dozen items using Planning Poker.
One may argue that while Magic Estimation saves you a lot of time, you might be losing in accuracy as opposed to Planning Poker. It's a valid concern but here's the paradox. Conventional wisdom has it that estimating large backlogs with Planning Poker is slow as hell and thus impractical, so nobody does this anyway. Surprisingly, estimating iteration backlogs with Planning Poker is no more practical. On the scale of two weeks the error margin is a few user stories at the very worst. When so little is at stake, there's no rational reason for perfect estimation accuracy. Instead, it's better to turn your focus and energy to breaking work down into smaller shippable pieces and making the team deliver truly working software at the end of each sprint — that is, actively learning and reducing risks.
Prepackaged with Relative Sizing
The part that I love about Magic Estimation the most is its visual nature. It makes it almost impossible to avoid estimating relative. You are literally forced to compare items against each other by having to place them on the board either left or right. That's the best way I've found so far to quickly make people understand relative estimations.
There's a a reason why I think it works. It's typical to use the (slightly modified) Fibonacci sequence in relative sizing. Human brain is much better suited to comparing on a logarithmic scale [6] and the Fibonacci sequence is an exponential function that makes one (strictly speaking, it becomes one if interpolated to a continuous function) [7][8]. Magic Estimation is in fact a heuristic that leverages this fact. The idea is simple. If one can spot a difference in size between two adjacent items, it is likely that those items are approximately in relation to each other as the corresponding Fibonacci sequence numbers. Because if they weren't, the difference wouldn't be noticed and the items would end up being the same size.
Conclusion
Planning Poker is a technique that still has its use and is preferred by many. In my own practice, I found something else that provides me with more value and it is Magic Estimation. Here's a quick comparison between the two.
What Planning Poker is good at:
What Planning Poker is good at:
- facilitating deeper discussion and risk identification.
- saving time,
- producing good enough estimates at a much lower cost,
- promoting the right understanding of relative sizing.