The UX designers at Envato are embedded within agile development teams. This means design has adopted methods for measuring our performance from our agile practices. However, alongside our engineers and product management colleagues we are working to change this. We believe it is also really important to measure the effectiveness of our work.
What you measure determines what you achieve
How you measure a team sets how a team behaves and how well it performs. A team will naturally optimise themselves around achieving the targets they set.
I believe this is where most agile practices go wrong.
A team will naturally optimise themselves around achieving the targets they set.
Agile is all about the delivery of value into the hands of the customer. “What’s wrong with this?” I hear you ask. Well, the problem lies in the fact that people focus only on measuring the delivery part. They ignore the value part or the impact that value has on the customer.
For example, Scrum measures velocity, burn, and story points. Kanban measures cycle time and flow. All these measures celebrate the act of shipping software to the customer. So the teams will naturally optimize themselves around shipping. High-fives all around.
When you measure delivery you expect deliverables
With the whole team focused on the shipping, the pressure on the designers is huge. Designers are expected to deliver the deliverables of design—wireframes, mockups, prototypes, and spec documents. And they can’t let the development team slow down. This is where most agile design teams get stuck, feeding the development beast.
We’re stuck in delivery mode because all we ever measure is shipping.
Jeff Gothelf, author of Lean UX, has been reminding us for the last 5 years to “get out of the deliverables business.” Yet we’re still doing it. We’re stuck in delivery mode because all we ever measure is shipping.
Changing the way we measure
At Envato we still value shipping, it’s still really important. We measure how fast we ship working software into the hands of our customers. As well as the design team ability to support our developers in doing so.
But, taking a data led approach we are now measuring the impact that the value we ship has on our customers. The teams are starting to measure key business metric that matter. They focus on making a difference to these metrics in every two week sprint. A success factor or hypothesis is decided at the start of an initiative. This defines when we are done, not just when the feature ships.
For example:
- We measure the number of people that successfully sign up against those that drop out. We’ve seen significant improvements since we designed new ways to sign up such as during checkout or by reducing the number of steps.
- We looking for an increase in conversion as we iterate on the way we communicate the benefits of our items—such as support
- We are gradually reducing our page load speed. We discovered a link between slow page load speed and high exit rates. We believe this results in fewer purchases on those pages.
Because our designers are embedded in the development teams, they share the same metric-driven goal. So everybody on the team—not just the designer—works on making a positive difference to the impact our designs have on the customer.
This article was first published on InVision Blog: How do you track your design’s effectiveness?