First of all, it’s been far too long for me writing something about metrics. It’s one of those topics in the agile community that keeps on giving ;-) 

But I’ve been inspired (yet again) by an article that Anthony Crain wrote a while back on the topic. He introduced it as his QPPE Metrics Model, where:

Q – represents Quality,

P – represents Predictability,

P – represents Productivity, and

E – represents Engagement.

You can find the article here –

https://techbeacon.com/how-software-teams-can-measure-anything-qppe-metrics-model

I shared this article with my friend and colleague, Shaun Bradshaw. I’ve known Shaun for the better part of two decades and he’s my go-to guy when it comes to all thing’s metrics related. Here’s his initial reaction to the QPPE post:

Bob,

Interesting read and thanks for sharing. I see that this aligns, in many ways, with the four-quadrant metrics framework we typically discuss. A couple things I like about it:

  1. There is an order to what should be measured first, second, third, and fourth. It makes sense to me that we want teams / orgs to solidify quality and predictability before emphasizing the other measurements (at least, I think his argument holds water).

  2. The idea that “value” should be a part of the “quality” metrics. As I was reading his reasoning for this, I was reminded of Cem Kaner and James Bach’s definitions of software quality, which both incorporate value to the customer (or someone who matters) as part of their definition. For that reason, I could see updating our quadrants to have “value” as part of quality and incorporate productivity as the 3rd quadrant.

There are a few things I question though, which is simply to say, I think there are some improvements that could be made. First, while I like the idea of focusing on quality and predictability early, I do think there needs to be some balance in all four-quadrants. In other words, would a focus on quality and predictability lead to gold-plating or spending too much time on estimation? Could it negatively impact team health or, as he puts it, engagement? Would the desire to maintain predictability make a team less likely to want to experiment?

Second, I’m not sure I really like his primary metric for Productivity - MTBR. The problem I see with this is it doesn’t necessarily capture costs. MTBR seems too heavily focused on time and scope. In other words, was the reduction in MTBR a result of standing up 1 or more additional teams? With that said, he does show a dashboard that incorporates measurements of Scope, Time and Cost.

Because Productivity is multi-dimensional in nature, I think it pretty much requires a multi-dimensional metric or you have to ensure that at least one of the dimensions stays constant. I’d have to think through it a bit more, but if an organization maintained a steady release tempo then you “fix” the time dimension and would only need to measure Scope and Cost at that point - in many ways this is similar to having set sprint lengths and measuring a team’s “productivity” by tracking velocity.

Third, he seems to short change the Engagement aspect of the framework. I like our “Team Health” a bit better, although it might be helpful to incorporate some of his “engagement” concepts. Perhaps measuring retrospective outcomes and/or team experimentation as part of team health (although you may already see those a part of the “agile behavior” or “functional behavior” aspects of team health.

I thought Shaun did a really balanced job of reacting to the QPPE framework. As I read his comments, it dawned on me that I/we haven’t shared our 4-Quadrants model in a post. So, here you go…

4-Quadrants of Agile Metrics 

I’ve talked about this before in another blog post around 2012. You can find it here - http://rgalen.com/agile-training-news/2012/6/2/the-agile-project-managerthe-essence-of-agile-metrics.html

However, I didn’t really pack the ideas into a framework. Much later, around 2016-2018, I started talking about the 4-Quadrants of Agile Metrics. The focus was on having a balanced set of KPI’s around:

  1. Predictability

  2. Quality

  3. Value

  4. Team Health

In creating and measuring your agile culture. In many ways, we align with what Anthony is saying. But as Shaun said, I think we focused more on Team Health and (the Team) than Anthony described.

We also focused away from velocity or output-oriented metrics for Predictability (thus the term) and more so focused on predictable planning and forecasting as the goal.

Wrapping Up

First, I want to thank Anthony for contributing his thoughtful model. I personally don’t want to critique it as I think Shaun handle that beautifully. But no matter how you view it, Anthony did a great job in pulling it together and sharing his thoughts publicly.

Second, I want to thank Shaun for his feedback.

Finally, I want to encourage everyone reading this to consider these meta-factors when you’re considering HOW to measure your agile efforts.

  1. Balance across whatever your measurement factors is critical.

  2. Focus on outcomes over output.

  3. Keep your people front and center.

  4. Connect to your customer (impacts, delivered value, satisfaction/delight) as closely as possible.

Stay agile my friends,

Bob.

 

2 Comments