by Chris Cardello on May 12, 2012

Being a consultancy, we at Hashrocket are frequently presented with the challenge of accurately setting budgetary expectations for our clients. Traditionally this has been handled by capturing requirements via story-cards and then estimating the complexity and risk of those stories using a point-based scale. We would then sum the total points for the project and divide that by our historic average velocity to project the amount of time we think it might take to complete the project. Over time we have learned how absolutely flawed this system is.

The main reasons we have found this to be an issue are:

• We have no idea how complex something which is going to be implemented in X months time is actually going to be. In fact, the more time that passes after a story is captured, the less an estimate is a reflection of the actual risk and complexity associated with a given story.

• As we (project and client teams) are always learning more about the domain and application, features are subject to substantial change between the time they are captured and the time they are implemented.

• The stories we write in the latter part of a story-carding session have a high probability of not being in the final scope of the project due to changes in desired functionality

Before you can even start development on a new project the client is going to need at minimum a ballpark of time to market and cost. At Hashrocket there has been an ongoing joke that estimation is pointless because “Every story is 2 points”. In reality, a closer look at past projects shows that in most cases they ARE all 2’s. Reviewing the past four launched projects I personally have worked, the average points per story delivered are as follows:

• project a: 1.69
• project b: 2.14
• project c: 2.09
• project d: 1.96

Historically the stories were assigned point values to help communicate complexity and ensure that the features being described were broken down to the smallest deliverable piece of software. More and more we found that stakeholders were making a direct correlation between the point values associated with the story and either time or money. We knew it was our fault, without knowing it we had essentially taught our clients that the way to measure the health and success of a project was to look at the points being delivered. The real measure of success and health should be whether we are delivering features the client cares about. When estimating scope for a client its just as easy to look at the story count as a reflection of the scope.. just assume they are all 2‘s. After all it’s only an estimate and no matter what, any time or budget projections we provide are intended to be interpreted as educated guesses.

We have found that it’s much better to do just in time estimations during Iteration Planning Meetings to make client’s aware of complexity and risk associated with a given story. During story-carding the idea is to try to get a minimal slice through the entire application. Iteration Planning Meetings are where the magic happens. This is where we fill any gaps in stories, talk through complexity, ensure acceptance criteria is still accurate, uncover hidden scope, and also challenge the relevance of a given story’s place in the backlog. If we estimate during our Iteration Planning Meeting the client is immediately aware of the riskiness of a given feature. They also have context that better equips them to make the decision as to whether a story is valuable enough to be in scope at that time. They know how much they have spent thus far, how much they have left to spend, and how much scope remains. Also, they have been working with us for however many weeks or months at that point and are more familiar with what something means to be risky. Up front estimation and scope discussion directly after story-carding are difficult for a client because at that point it is hard for them to detach themselves from the big picture of what they think their application is and root themselves in the reality of what is actually capable of being done within their budget.

Velocity. It’s not as dirty a word as some might have you believe. Hate it or love it it’s the best option for reflecting what has been delivered. Sure it’s easy to look at a number and conclude that things are going poorly or well. I like to remind my stakeholders that when looking at velocity you have to remember that it’s always a representation of what HAS been done and is not necessarily going to accurately project how much WILL get done in the coming weeks. Also, it’s our job as the project team to communicate with the client all of the different factors which can impact velocity from week to week, and stress that to really measure the health of the application.. log in, test it, and exercise the features.

When kicking a project off don’t pretend to know how complex or risky every story in the backlog is going to be. As the majority of stakeholders are not going to be technical, take the time to educate them. Be informative as to the factors that can impact delivery over the life of a project. Explain why estimating into the distant future is not reliable. Provide examples on previous projects where something which was initially thought to be simple ended up being a 3 week endeavor, and vice versa. This isn’t to say you shouldn’t provide ballpark estimates or points of comparison when attempting to communicate complexity, as it is vitally important to communicate openly with your stakeholder. It is to say that setting expectations at a point where you are not informed enough to know the full scope of what you are estimating is essentially lying to your stakeholder.