Devlivery Metrics

,

Delivery Metrics

Delivery metrics are critical to understanding your teams’ performance - but, this truth stems from something much deeper and important: value. Delivery metrics are a model teams use to undertstand the delivery of value, often before that value materialises into a terminal outcome (quite often financial).

The common stats aphorism “All models are wrong, but some are useful” holds true for delivery metrics too - so more than just finding the right metric, as an engineering manager, you are looking to create an environment where squads seek to continuosly improve their understanding of value delivery.

This is both non-obvious and materially difficult. There are two reasons for this:

  • If a team or organization’s key metric definitions are changing over time, it becomes difficult to actually track results over time, primarily because the metrics they have aren’t comparable over time
  • Working on defining value and working on delivering value at the same is not a natural thing to do - done poorly it adds confusion, slowing down or halting actual delivery

So, what are delivery metrics?

Let’s start with a metric. A metric is a unit of measurment that quantifies a property in a phenomenon being observed. A good metric should be:

  • accurate and valid : a thermometer that tells you the wrong temperature is of little use. If it tells you the correct temperature but you don’t understand why (the principles behind the measurement) it’s quite dangerous - akin to an oracle more than a tool
  • reliable and objective : given the same conditions, you should get the same results and importantly, these should be free from bias or subjective interpretations, providing an objective measure of the phenomenon being observed
  • relevant and actionable : the metric should measure something that is meaningful and relevant to the problem being solved or the decision being made and provide insights that can be acted upon - no use in measuring things you have no control over
  • consistent : the metric should be consistently measured and reported in a way that allows for comparison and analysis over time

What about a delivery metric?

A delivery metric at it’s best should measure the value created by a team, in a way that can be used to understand the team’s trajectory over time. It should help you understand your team’s performance and act as an important tool to understand the state of your team and thus, any interventions that can help you improve it.

In engineering management, story points, burn down charts, number of defects per release, cycle time are all examples of measuring proximate outcomes - they measure the exhaust of the development process. I think these are so popular because measuring impact can be so hard.

Software development metrics can be very useful but on their own not enough. Just like ship goals, they make it easier for you to confuse motion with progress - but absent of any other metrics to measure impact, they can be incredibly valuable.

So, what to do?

First, make sure you have some metric. This sounds obvious but literally every place I have worked at, I’ve found at least one team that doesn’t track the value they intended to deliver. You can start with software dev metrics, but be sure not to end there.

For some teams this will be an uncomfortable process. This can be for a number of reasons, two of which I’ve found most salient:

  • it’ll force a team to characterize the value they intend to deliver and in doing so uncover some hard truths: poor definition, incoherent or lacking mission, low value
  • anxiety stemming from understanding there will be an objective tool to evaluate the team’s performance

In a kind and empathetic way, push through that lack of comfort. It’ll be better for the team and the individuals within that team. Ultimately feeling like you are impactful is a powerful instrinsic motivator, and being able to give clarity to that impact will pay dividends.

1 Resist the urge to give your team the right thing to measure.

Unless it’s time critical

It presumes you know what that thing is in more resolution than your team. More importantly, pushing a team to use metrics will only be partially successful, having a team demand the right metrics will make a team exceptional.

Ultimately you want a team that deeply understands and is motivated by the impact they can have and is intelectually honest about that impact.

  • “motivated by the impact they can have” = can build a vision of what the future looks like once impact has landed and is motivated by that future
  • “deeply understands”: has built a model that can be well characterized about what leads to that future beyond narrative. That is, can describe a set of expectations and hypothesis that lead to that future

2 Build an environment where your team, and especially leads within it, actively seek to find and validate their expectations and measurment of value.

How do you do this? Most often through coaching - the socratic kind. Specifically through questions, about intended outomes, about hypothesis, about alternatives and cost of opportunity. Help your people discover that autonomy and independence most often come from trust and an effective way to build trust is to leave little doubt that the team care about the right things. Help your people understand that ultimately you trust them to make the right decisions and good decision making is informed decisiong making.

3 Provide tactical support and air cover and intervene when you need to.

As the team considers (and implements) what the right thing to measure is, it’ll likely need to make changes, either in the outcome it’s measuring or in the team’s direction as it learns about better pockets of value. This is where you can provide support: making sure your leadership understands why and how things are changing and making sure your team has the time to show progress. With your proximate (software dev) and main metrics, intervene. This whole exersice is meant to help you help the team continually improve in a grounded way, measurement is the fisrt step.