Large Batch Death Spiral

This is a term I first come across in Don Reinertsen’s “Principles of Product Development Flow” book (“B9: The Batch Size Death Spiral Principle”), but I’m constantly reminding myself of its importance . Essentially, this describes the behaviour of how large batches with their infrequent delivery seem to attract more and more work – becoming even larger batches. As larger batches take longer, there is an increasing problem of dates moving out and more and more work being added. Essentially this can become a never ending spiral of work that never gets done.

Essentially, as projects start to get larger they invariably create a “gravity” to which other things are attracted to, creating even larger batches. As things start to slow down, other things become dependent on the large project. Given that it’s really large with a lot invested in it, often this project becomes the priority with others having to give way to it. However, this is not always the case – other pieces tend to “attach” themselves to the large project because this is the only way to get things done.

Once a few things have attached to the large project, it makes that project even larger. Timelines start to blow out and everything slows down. Once this avalanche has built up enough momentum it’s almost impossible to stop. With all the interdependencies and things hanging off the key project, it’s hard to start to unwind it. Now this key project becomes the only available delivery vehicle to the organisation and other things start to attach itself to it.

As things get critical, people start to work late nights and weekends trying to get that key project over the line. Although burnout might not be specifically mentioned, it’s likely this is happening to your staff. They get sick as a result of the workload and in the sum total is that you haven’t gone any faster. People get unhappy and leave the organisation further impacting timelines and puts more pressure on those who remain.

Now you’ve invested a lot of capital into this and in some organisations, by cancelling a project they convert the CAPEX into OPEX and that effects their budgets to “keep the lights on”. That project is now not contained within itself, it’s starting to impact your business at a fundamental level.

Often nobody in middle management wants to take the onus of stopping such a large investment and it can be a career limiting move. There may not be enough safety in such an environment to make such a call and thus the reporting continues to be positive. Given that delivery gets delayed there’s often little negative feedback as it’s coming from internal sources, rather than real customers further fuelling the biased feedback loop.

Executives may not be aware that there is a ticking time-bomb waiting for them – at some point, it may even be years down the track, they will have the shock of discovering everything isn’t as rosy as they had assumed. The result can often be that they need to go into full damage control mode – sometimes they may even have to act quickly to save the business.

To avoid this kind of situation, you will need to use WiP limits and other means to control batch sizes. A project is essentially a batch of work, so understanding how these batches work is key to unlocking the batch sizes. Just putting a WiP limit on the number of projects may not be enough – if one project is massive, then you will need to look further below to see if you’re really going to be affected by this.

One way to prevent it is to only release funding on a smaller and limited increment – for example per feature. This kind of constraint places a focus on the transaction costs of large batches and tends to start to unwind the damage caused by the large batches. You may even impose time based constraints – eg every project must delivery some value at least once a quarter / half financial year.

Other ways are to treat batches over a certain size as critical risks that must be dealt with. Often there are ways to decouple items or reduce transaction costs / time of delivery such they don’t have so much to do all at once.

Instilling an air of safety in the organisation that allows managers to give full transparency to executives and be honest about the options. Having up front upper / lower control limits defined will help you guide this conversation objectively. Being able to stop this train wreck before it happens will be key, often it’s easy to start projects, but they are really hard to stop – adjusting your process so that you’re able to stop will be useful.

Large batches can quickly spiral into larger batches. These can be a critical threat to your business and it may not necessarily be immediately apparent. These can be avoided if you put in place measures and constraints to control this kind of situation.