The Engineering Metrics Dashboard at SV.CO

We recently built an engineering metrics dashboard at SV.CO. On a single page, it tracks nine things that are like health statistics for a project. In addition, it provides me as an engineering lead, a bird’s eye view of our team performance.

Here’s how it looks like:

Engineering Metrics Dashboard

This is what we track

  • Deploys: The number of deployments per day. Because we use continuous deployment, this number is often north of single digits. What I look for in this stat are long stretches (2 weeks+) of stalled deployments: this often means that we’re not breaking down work into small units, and it’s time I stepped in to take a look.
  • Bugs: The number of bugs reported per day by the team. We want to improve this with the number of bugs fixed and also include in our production automated error reports. This stat, which a slightly lagging indicator of code quality, is still useful especially immediately after our feature sprints. At SV.CO, we try to go by a tick-tock cycle (with the tick representing new features, and tock the needed stabilization or bug-fixes necessary for those features), and examining this stat after a tick cycle will provide a good thumb-rule for the time necessary for the tock.
  • Git Commits: This metric tracks our velocity. As all our stats, it’s not a perfect indicator (especially when team members pair off), but it’s a good way to track slow periods. Since we emphasize atomic commits, it’s also a warning flag when a big feature lands with a lesser commit count.
  • LOC Change: I’m a strong believer in the adage that the best code you write is the code you do not. Or better yet, the code you delete. So we track both LOC additions and deletions. And we celebrate the code that we throw away.
  • LOC Language Split: This is a companion metric to LOC Change, and the one thing I track here is removing older deprecated languages (like CofeeScript).
  • Test Coverage: is a good useful indicator of code quality. As a development shop, we’re not anal-retentive about TDD or unit testing, but we do like to cover features with good integration test coverage. This usually means Selenium-style testing with capybara.

While building this dashboard, we kept several common criticisms in mind.

  1. These are vanity metrics. I’ve explained the usefulness of these metrics above, but this requires a bit more explanation. As a human being, I don’t see a problem with taking pride in the work we do. And as a software engineer, most of the work we do happens to be ephemeral: we work with building blocks that do not stand the test of time. What we can be proud of constantly is the programming journey. So I call these metrics what we can (or should) be proud of.
  2. It’s hard to keep them updated. What we’ve done is hooked them up to APIs that our services provide. For example, the code commits are pulled directly from Github, and the Bug reports are pulled off our Trello boards. This still will require maintenance but hopefully far less than if these were manually tracked.
  3. Why don’t you track these data where they originate? The answer to this is pretty simple: we use quite a lot of software to manage our work, and it’s nice to have all the data in one place.

That’s it for our dashboard! Any feedback appreciated.

 

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s