Story points vs Man days – The misguided debate

Relative estimation and story points is one of the topics I find people most often struggling to grasp, whether in trainings or at client sites. The main issue seems to be the belief that eventually, Story Points (SPs) need to be translated into Man Days (MDs) if you want to be able to do things like capacity planning, estimation and portfolio management. And because of this, some people have a hard time understanding the real reasons for using relative estimation. And even worse, their focus on the abstract concept of MDs prevents them from seeing the bigger picture and what really matters – value.When I discuss this topic with clients, I always try to highlight that there are two distinct issues at play here:

• how relative estimation can be plugged into any existing MDs driven process
• how the focus on MDs clouds managers from seeing the real issue

Using relative estimations in a MD-driven organization

The MDs currency is a constraint that Agile coaches typically cannot avoid when working with clients. Often, the organization’s entire planning & budgeting structure is based around MDs and changing that structure is not in the scope of the Agile change initiative (for now). Rather, managers want to know how they can use relative estimations within this MD structure.

After going through this exercise with most clients I’ve worked with, I’ve found the easiest way to explain how to do this is by writing 3 simple equations on the whiteboard, like this:

I explain that the first line (estimation x factor = effort) is the formula that everybody uses to obtain MDs, regardless of methodology or discipline.

The second line is how that formula works in the Waterfall world, where a team will estimate the requirements in MDs and the manager will apply some conversion factor to account for the non-productive time and overhead of the team, to finally obtain a MD figure that they can use for budgetting and planning.

The third line is how you can obtain your MDs when using relative estimation. In that equation, the velocity factor = MDs per Sprint / Velocity.

Fine, no issues up to here, it’s very straightforward and managers get the equation. But this is where the questions start. In a recent case, a team manager was challenging this model, and his crticism focused on two basic points:

1. The “velocity factor” (MDs p/ SP) is essentially an exchange rate for MDs. So if the team does not have a consistent velocity (which was the case for his Scrum teams), this exchange rate fluctuates too much meaning your estimates will likely be incorrect.
2. Story Points are too abstract, and since the two formulas are essentially the same thing, why not just estimate in MDs?

Both criticisms he raised helped me understand the rootcause of the disconnect.

I explained that in issue # 1 (fluctuating velocity), he was absolutely correct. If your team’s velocity is fluctuating a lot, your MD estimation using the conversion formula will indeed likely be incorrect. Never forget that Scrum does not solve your problems, it just makes them painfully visible. And that is exactly what was happening here.  You still have to do the hard work of fixing them.

He should be talking to his team about what are the reasons that their velocity is fluctuating so much and listening attentively to their feedback. Very likely these issues have already been raised in their retrospectives (which was the case here). The manager’s focus should be in removing these impediments, helping the team deliver more consistently, instead of trying to somehow magically improve their ability to estimate.

And this was the perfect lead in to address the second issue – “story points are too abstract, why not just use MDs?” I went back to the whiteboard and drew two red circles:

Yes, the formulas were very similar (they are both equally simplistic), but the difference was on where the focus was placed. In the waterfall version of the formula, there is no consideration for productivity. Its focus is on getting the estimate correct, since the overhead factor is easy to calculate and doesn’t fluctuate much.

On the other hand, the Agile version of the formula flips that focus away from the estimate. Relative estimation is not difficult and can be learned in 1 hour. Besides trying out different relative estimation techniques (team estimation, planning poker, …), there isn’t much to improve there. Rather, it is the velocity factor that we focus on. That is a measure of our productivity, and if that factor is changing wildly or trending in the wrong direction, Agilists want to find out why. We’re trying to unearth the actual problems that are keeping us from a sustainable, productive pace.

(note: velocity is an imperfect measure of productivity.  Using it is a barometer for the Team’s delivery capacity and as a data set for retrospectives is very helpful, but trying to use it as a performance goal for a Team is missing the point completely.  As with any imperfect measurement, velocity can be gamed, so don’t lose time trying to use it as a performance metric for the Team.)

If your team’s velocity is too unpredictable, then no formula in the world is going to change that fact.  You need to get your hands dirty and find out why that’s happening.  Often times, it will be related to the fights managers are trying to avoid for political reasons (dependencies on other teams, inconsistent test data at the corporate level, bad product management, bad technical practices, …).  If you want to improve your delivery capabilities, start tackling those impediments.

Focusing on MDs is missing the point

The reason MDs are so prevalent on the mind of managers is because this is the accepted currency of IT organizations. In fact, it is so common place that managers often forget it is not the end goal, but rather an abstraction layer used to represent the hard-to-measure-yet-always-mentioned Business Value.

Side note: while in the corporate world, business value will usually mean profit, it (value) does not have to equal money. It varies depending on the purpose of your organization (customer happiness, lives saved, quality, …).

Instead of trying to think about how to measure Business Value, managers feel comfortable in the MD abstraction layer.  This leads to misguided success metrics for projects, such as “deviation from initial estimate”.  These only perpetuate the focus on getting that damn MD estimation correct and cements the erroneous belief that eventually, everything must be translated into MDs.

I say this debate is mis-guided because I’ve seen many managers lose sight of what they should be focusing on. I’ve heard many IT managers tell me that their number one priority was making sure that they delivered projects on budget (MDs). Not improving the throughput of their teams, not reducing technical debt or time-to-market. No, their priority was nailing the estimates.

Essentially, they’re saying “I don’t care if we’re delivering cr*p, I just want us to deliver cr*p in a predictable manner”.

This is a problem. And until managers are willing to move beyond this focus on being predictable over being productive, they are essentially becoming an impediment to the improvement of their teams. Because it’s only natural that their teams will sense the focus on predictability over value creation, and they will make it their priority also.

The mindset shift that must happen is the realization that the focus should be on delivering value. And to achieve this, even SPs are not sufficient, they only measure the amount of work a Team is able to deliver. Organizations looking to improve their ability to deliver value, must first figure out how to measure it.

In fact, one can easily imagine that it is precisely the inability to measure value that makes managers, instead, focus on the cost side of the equation.  It’s hard to say how valuable a story (or even a project) is, but calculating how much a project deviated from its original estimates involves little more than 6th grade math.

The #NoEstimates movement has been making a lot of noise about this recently. Vasco Duarte wrote a good, short overview of it, for those interested.

I don’t disagree (my favourite double negative) with anything they say, except that I don’t like the name (everybody estimates, even in the scenario they are proposing) and also I don’t think they are describing any breakthrough, but rather an advanced state for organizations applying Lean Thinking.

I prefer to think of estimations as Waste. Better estimations are not what will make an organization successful or help you deliver your new, super-cool product to the market. But even though it doesn’t add any real value, it’s inherent to product management and software development. Minimizing this waste is a gradual and never-ending journey.