[2020 devnexus] metrics that matter

Speaker: Joel Tosi @jdojoandco

For more, see table of contents


General

  • Finishing on time doesn’t matter if do wrong thing
  • Can’t just say happy; need to know how much costs
  • Who cares about metrics: managers, product owners
  • People tend to measure what is easy to measure
  • Goal is to make people happy (and make money)

Bad metrics

  • # people trained as measure of agile transformation
  • measuring just a piece. Doesn’t matter how fast deploy if deploying wrong thing. Sub-optimizing

Simple metrics

Good place to start if have no metrics now. (but not always good metrics)

  • Number of defects
  • Velocity
  • Lines of code
  • Number of teams (not x% transformed b/c y teams have done it)
  • Code coverage. (bad if game system: ex: tested all getters/setters to increase)

Notes

  • Easy to collect
  • Doesn’t tell customer impact, right or usable
  • If don’t have insight, can help

Directional Metrics

Take time to capture

  • Increase in code coverage
  • SQALE (from Sonar)
  • Reduction in % of defects
  • Cycle time
  • Deployment frequency
  • Average time a bug takes to get fixed
  • Number Checkstyle violations over time
  • Size user base

Notes

  • More sustainable. Ensures not killing team in meantime
  • More depth
  • Still not measuring impact to users

Impactful/Economic Metrics

  • Reduction of cycle time for a delivery that mattered
  • Systemic cost reductions
  • Stopping bad ideas

Notes

  • How do we get rid of noise in the system
  • People need to agree to do these types of metrics
  • Need psychological safety. Must be safe to be wrong

Process Behavior Charts

  • You deliver value with a system.
  • A stable system will continue to deliver outputs within a range if you do nothing different.
  • Goal is not to react to noise.
  • Process Behavior Charts help separate signal from noise
  • Need to understand what measuring
  • Chart data in a time series
  • Calculate the moving average
  • Calculate the upper/lower bounds by multiplying by 2.66
  • Introduce change and continue to measure

Supporting Knowledge Work

  • Better decisions if faster access to info
  • Onion – team, manager(s), business, execs. Who can answer a question. How many levels need to go through

Books

  • Measures of Success: React Less, Lead betteer, Iprove More – Mark Graban. Covers health care.
  • Understanding Variation – The Key To Managing Chaos – Donald J Wheeler. Explains why 2.66 and more complex charts
  • Principles of Product Development Flow – Donald G Reinertsen

My take

Good talk. I like the interactivity to supplement the slides. I was thinking it would be more concrete. But I liked what it was actually about. I hadn’t seen process behavior charts before and found that particularly interesting.