Value Team Heuristics and use of Story Points when forecasting

When it comes to estimation in Software development, the difficult question to answer is when can we go to market? and when can we release the product or service? These are difficult questions, both from the business perspective as well as the development perspective.

There has been a lot of conversation in the agile community around User Stories and use of Story Points to size the product backlog items. What is often lost when teams adopt this technique is the value of team heuristics, as in team experience and the importance of relative sizing.  These put together and followed with some discipline allow the team with the ability to forecast – as in over discrete periods of time, sprints,  express an ever accurate answer to the questions:

How much by this date?

or

All this by when?

If no one is asking these types of questions at the end of each sprint then you may be in danger of falling into the cargo cult scrum trap, as in following a ritual of sizing product backlog items for ritual sake.

So what does heuristics really mean?

In a nutshell heuristics refers to experience based techniques for problem solving, learning and discovery, one which is good enough approach to finding an answer, where the perfect answer is not knowable or would be too expensive to learn.

So, if you are familiar with how most sciences work, you have probably encountered the use of  heuristics before. For instance, rather than predicting exact ratios of chemicals for a reaction, chemists use heuristics as guidelines in predicting how various chemicals will react. Similarly in engineering for instance aircraft designers use heuristics by considering coefficients of lift and drag to point them in the right direction, with the design empirically refined based on experimental evidence.

In essence to answer either of the questions stated above there is a heuristic alternative to the careful reasoning that occurs in the long cycles of phased approach in gathering requirements, analysis, design, development and test. This heuristic alternative works well for estimation and this is in spite of sometimes serious errors. The key is to ensure the errors are bounded across short time intervals, with frequent pauses assess the outcomes and inform what to do next. Of course Scrum provides for a natural cycle of pauses at the end of each timebox, where the team can take stock of the outcomes as well as assess the steps the team actually took vs. assumed what they took, yes essentially these being the sprint review and end-of-sprint team retrospective.

First Order Estimation

So what is the first order of estimation, well it is something we all do so naturally and effortlessly when it comes to driving a car to riding a bike to walking in a busy mall, else surely we would be running into each other most of the times. I mean when is the last time you got out a measure tape when changing lanes on a freeway or overtaking to ensure you have the most accurate measurement in distance or speed.

In essence the first order estimation I refer to is a relative measure, in this case of work items (user stories) to one another.  To me it is similar to coefficients of lift and drag considered in aircraft or even car design. Its just that in this case these estimates can be now be thought of as being the teams coefficients, one that corresponds to the team that is going to do the work based on their perspective for product backlog items as opposed to anyone elses.

Team’s coefficients

There are teams that only look at their team coefficients, user story points, when planning a sprint. This may well be all that is needed for planning, but in truth this is only the case for a teams that have been together for a long time, have experience of working with each other and have established team heuristics, their rules of thumb.

Teams that are new or new to working as a scrum team often miss out the need to assess the team capacity at a more granular level of planning, one that breaks a product backlog item (user story) into actual tasks that need to be performed in order to get to done!, before making team commitments. So it is best for a team to be mindful and disciplined towards developing the team heuristics and continued reliance on wisdom of the crowd to decipher difficult problems that includes estimation.

To do this start with having a stable team one that takes a disciplined approach in their scrum ritual meetings during the sprint. This includes when refining backlog items and/or sizing using story points to when planning out by defining tasks along with task level time estimates as this is what will help shape the teams commitment to the plan that appears in the form of a sprint backlog.

Eureka!

Daniel Kahneman, in his book Thinking, Fast and Slow, points to how judgement happens, and while it is a complex function of the brain what he points to is what Psychologist have to offer based on their observations supported by what Neurologist tell us about functioning parts of the brain.  In essence we have evolved to with two modes of thinking, called System 1 and System 2 as coined by psychologist Keith Stanovich and Richard West.

Where System 1, evolutionary the oldest, the part that is responsible for fight or flight and ingrained to our survival instinct, operates continuously and generates assessments of various  aspects of the situation with little or no effort. This basic assessments plays a vital role in intuitive judgement, as this routinely substitutes the more difficult question being asked of by System 2, the essence of heuristics.

Where as System 2 thinking considers, only if it must, the difficult to answer questions the one that System 1 doesn’t readily offer an answers to, and it triggers many other computations including basic assessments acting as a mental shotgun according to Kahneman. He points out the word heuristics comes from the same root word as does the word eureka, and is technically defined as a simple procedure that helps find adequate, though imperfect answers, to difficult questions.

Wisdom of the Crowd

So the when it comes to planning for a release or a sprint, the team takes over the role of being the mental shotgun that Kahneman refers to  by seeking to establish a shared team understanding of the problems selected in from the product backlog.

When it comes to the question: When do you think we can have this item by? For ease in cognitive load not to mention the impracticality with undertaking an exhaustive search, one can speed up the process by relying on team heuristics to find a satisfactory solution that we know isn’t guaranteed to be optimal by answering an alternate set of questions, such as:

  • Does this item resemble in level of domain and/or technical uncertainty like any other problem we have already seen on the product backlog? 
  • Does this item have greater or lesser domain/technical uncertainty?
    • Does this item offer higher/lower levels of complexity than other items recently encountered?
    • Does the item offer a high order of parameters to optimize a potential solution over?

In essence having the team size the product backlog item is the wisdom of the crowd, as when it comes to estimating where you have to give considerations to multiple parameters is a difficult problem! so much so that individuals do poorly whereas pools of individual judgements do really well. Common examples of this method that we all know of include using a rule of thumb, an educated guess, an intuitive judgment, stereotyping, or common sense, in the case of a team its the case of cultivating team’s intuitive judgement and rules of thumb.

When it comes to leveraging wisdom of the crowd during estimation, it works well if the participants adhere to principle of independent judgment. The reasoning for this is, errors that individuals make are independent of the errors others make and in absence of systemic bias (or shared bias) they tend to average out.

One popular technique introduced to scrum teams has been that of Planning Poker first introduced by James Grenning. This is a game lending to serious decision making, one where team members play two or three rounds to size each product backlog item considered. Players are guided to adhere to the principle of independent judgement for the first round against each backlog item. They are asked to make up their mind on the story point they would attribute to the backlog item and really have to rely on their individual heuristics and understanding of the problem statement. This usually relies on team addressing a set of the alternate questions to assert size of the product backlog item. Players reveal their hand of the round together, at which point they can exchange their assumptions, experiences and recollections. This helps the group clear up any common misconceptions and ambiguities for the problem statement. This no doubt now influences future rounds, and thus effectively helps the team reduce associated uncertainty and narrows the dispersion in sample size eventually.

According to Kahneman this form of group decision works if the participants make independent observations and their errors aren’t correlated. In essence, whatever errors there are will average out across the group and now you have the magic of error reduction. He does point out that allowing players to influence each other introduces shared biases which is not reduced in group judgement, meaning there is possibility of errors compounding. This may well occur from time to time,though not as often as you think, none the less you may recognize this when you hear something like this story was more like a 13 and no way was it a 5 from a team.  My guidance in this situation has been “great, now this is part of your team’s memory and its heuristicshindsight is like that. You’ll be able to use this when you encounter a backlog item that has the same hallmarks as this story”. In essence this will come out in the wash, more so if the team keeps to a shorter sprint cadence than a longer one. The reasoning is that such errors, if you will, are compensated for only if there is small delays with the feedback – the very think those pauses served by both daily standup as well as the end of sprint review and team retrospective in scrum.

As for the value of story points, that is often lost with teams as they may not have undertaken the activity of release planning. My guidance has been first suspend the idea of time, when looking at story points. Consider the story points as weights attributed to the stories by the team based on levels of complexity in the problem domain, degrees of unknown parameters you’ll have to find optimal solution on, unknown levels of technology and/or environment to account for when developing the solution. Alas I do recognize it is difficult to suspend this thinking, especially when dealing with numbers expressing relative size, and you have someone asking when do you think you’ll be done!

Summary

  • First off, team heuristics provides a reliable forecast for delivery, much more so if the team is setup as a stable unit than not.
  • Secondly, they are better off using story points and sizing their backlog items as a collective team expression.
    • This is an outcome that teams can attain during backlog refinement meetings, where they are getting together to honestly expose levels of uncertainty, ambiguity and unknown against otherwise build in assumptions stated in the story.

Teams that have not adopted or even abandoned story sizing are more akin to a blinkered horse in a race, as their ability to set expectations with the rest of the business functions (marketing, sales ops, business ops, support, tech ops, etc.) who also contribute to adding value to the product or service, will be truly compromised.

Sure, there are number of people in the scrum community who think any form of estimation and therefore forecasting is wasteful activity. Alas I am not one of them. As there isn’t any endeavor worth doing, that I can think of, in everyday life that isn’t budgeted some measure of time as worth investing for the desired outcome. This is then used to measure progress on and decisions relating to further investment in time and software development is no exception to this.

In follow up blog on this topic you can expect my observations of some common traps teams fall into when sizing their product backlog

 

Comments are closed.